Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.445707
Epoch 1
Validation prc-auc = 0.458308
Epoch 2
Validation prc-auc = 0.486724
Epoch 3
Validation prc-auc = 0.446652
Epoch 4
Loss = 5.9146e-01, PNorm = 78.5516, GNorm = 6.0189, lr_0 = 6.5000e-04
Validation prc-auc = 0.423135
Epoch 5
Validation prc-auc = 0.353443
Epoch 6
Validation prc-auc = 0.357471
Epoch 7
Validation prc-auc = 0.330555
Epoch 8
Validation prc-auc = 0.281123
Epoch 9
Loss = 7.6735e-01, PNorm = 78.7998, GNorm = 20.4609, lr_0 = 9.9743e-04
Validation prc-auc = 0.403514
Epoch 10
Validation prc-auc = 0.444280
Epoch 11
Validation prc-auc = 0.486078
Epoch 12
Validation prc-auc = 0.437510
Epoch 13
Validation prc-auc = 0.431015
Epoch 14
Loss = 5.1773e-01, PNorm = 79.1107, GNorm = 1.8584, lr_0 = 9.8890e-04
Validation prc-auc = 0.330359
Epoch 15
Validation prc-auc = 0.417203
Epoch 16
Validation prc-auc = 0.406734
Epoch 17
Validation prc-auc = 0.411512
Epoch 18
Validation prc-auc = 0.442178
Epoch 19
Loss = 5.3559e-01, PNorm = 79.4173, GNorm = 5.7605, lr_0 = 9.8045e-04
Validation prc-auc = 0.451489
Epoch 20
Validation prc-auc = 0.271484
Epoch 21
Validation prc-auc = 0.438021
Epoch 22
Validation prc-auc = 0.309709
Epoch 23
Validation prc-auc = 0.164991
Epoch 24
Loss = 5.1603e-01, PNorm = 79.7061, GNorm = 3.9063, lr_0 = 9.7207e-04
Validation prc-auc = 0.326223
Epoch 25
Validation prc-auc = 0.357101
Epoch 26
Validation prc-auc = 0.349736
Epoch 27
Validation prc-auc = 0.350833
Epoch 28
Validation prc-auc = 0.347300
Epoch 29
Loss = 4.4670e-01, PNorm = 80.0649, GNorm = 5.2925, lr_0 = 9.6376e-04
Validation prc-auc = 0.339963
Epoch 30
Validation prc-auc = 0.361679
Epoch 31
Validation prc-auc = 0.340161
Epoch 32
Validation prc-auc = 0.346327
Epoch 33
Validation prc-auc = 0.357100
Epoch 34
Loss = 3.6524e-01, PNorm = 80.4179, GNorm = 1.7099, lr_0 = 9.5552e-04
Validation prc-auc = 0.340565
Epoch 35
Validation prc-auc = 0.333382
Epoch 36
Validation prc-auc = 0.303599
Epoch 37
Validation prc-auc = 0.275057
Epoch 38
Validation prc-auc = 0.351124
Epoch 39
Loss = 3.5502e-01, PNorm = 80.7368, GNorm = 3.2371, lr_0 = 9.4735e-04
Validation prc-auc = 0.341097
Epoch 40
Validation prc-auc = 0.439518
Epoch 41
Validation prc-auc = 0.368610
Epoch 42
Validation prc-auc = 0.367481
Epoch 43
Validation prc-auc = 0.225364
Epoch 44
Loss = 2.5396e-01, PNorm = 81.1163, GNorm = 0.8848, lr_0 = 9.3925e-04
Validation prc-auc = 0.315748
Epoch 45
Validation prc-auc = 0.330290
Epoch 46
Validation prc-auc = 0.349946
Epoch 47
Validation prc-auc = 0.329768
Epoch 48
Validation prc-auc = 0.347256
Epoch 49
Loss = 2.4885e-01, PNorm = 81.5223, GNorm = 4.5164, lr_0 = 9.3122e-04
Validation prc-auc = 0.308439
Epoch 50
Validation prc-auc = 0.173983
Epoch 51
Validation prc-auc = 0.259395
Epoch 52
Validation prc-auc = 0.338578
Epoch 53
Validation prc-auc = 0.360054
Epoch 54
Loss = 2.4122e-01, PNorm = 81.9848, GNorm = 1.9906, lr_0 = 9.2326e-04
Validation prc-auc = 0.347880
Epoch 55
Validation prc-auc = 0.283884
Epoch 56
Validation prc-auc = 0.305573
Epoch 57
Validation prc-auc = 0.313327
Epoch 58
Validation prc-auc = 0.337140
Epoch 59
Loss = 1.5524e-01, PNorm = 82.4705, GNorm = 3.7498, lr_0 = 9.1537e-04
Validation prc-auc = 0.340310
Epoch 60
Validation prc-auc = 0.320154
Epoch 61
Validation prc-auc = 0.354776
Epoch 62
Validation prc-auc = 0.367880
Epoch 63
Validation prc-auc = 0.358730
Epoch 64
Loss = 7.5256e-02, PNorm = 82.8959, GNorm = 0.8970, lr_0 = 9.0754e-04
Validation prc-auc = 0.372602
Epoch 65
Validation prc-auc = 0.358680
Epoch 66
Validation prc-auc = 0.293166
Epoch 67
Validation prc-auc = 0.219673
Epoch 68
Validation prc-auc = 0.296784
Epoch 69
Loss = 7.5535e-02, PNorm = 83.2407, GNorm = 1.3195, lr_0 = 8.9978e-04
Validation prc-auc = 0.295494
Epoch 70
Validation prc-auc = 0.274386
Epoch 71
Validation prc-auc = 0.297512
Epoch 72
Validation prc-auc = 0.306965
Epoch 73
Validation prc-auc = 0.283575
Epoch 74
Loss = 9.9013e-02, PNorm = 83.6169, GNorm = 2.4787, lr_0 = 8.9209e-04
Validation prc-auc = 0.239506
Epoch 75
Validation prc-auc = 0.271515
Epoch 76
Validation prc-auc = 0.263982
Epoch 77
Validation prc-auc = 0.202906
Epoch 78
Validation prc-auc = 0.298382
Epoch 79
Loss = 1.0069e-01, PNorm = 84.0248, GNorm = 1.3521, lr_0 = 8.8447e-04
Validation prc-auc = 0.336569
Epoch 80
Validation prc-auc = 0.279503
Epoch 81
Validation prc-auc = 0.299879
Epoch 82
Validation prc-auc = 0.345763
Epoch 83
Validation prc-auc = 0.325699
Epoch 84
Loss = 8.1490e-02, PNorm = 84.4339, GNorm = 2.0025, lr_0 = 8.7691e-04
Validation prc-auc = 0.271849
Epoch 85
Validation prc-auc = 0.245776
Epoch 86
Validation prc-auc = 0.263068
Epoch 87
Validation prc-auc = 0.301349
Epoch 88
Validation prc-auc = 0.292092
Epoch 89
Loss = 5.0683e-02, PNorm = 84.8066, GNorm = 0.3284, lr_0 = 8.6941e-04
Validation prc-auc = 0.284103
Epoch 90
Validation prc-auc = 0.314465
Epoch 91
Validation prc-auc = 0.327746
Epoch 92
Validation prc-auc = 0.329459
Epoch 93
Validation prc-auc = 0.303492
Epoch 94
Loss = 5.3445e-02, PNorm = 85.0653, GNorm = 0.3499, lr_0 = 8.6198e-04
Validation prc-auc = 0.277749
Epoch 95
Validation prc-auc = 0.256772
Epoch 96
Validation prc-auc = 0.278746
Epoch 97
Validation prc-auc = 0.268610
Epoch 98
Validation prc-auc = 0.264428
Epoch 99
Loss = 8.9455e-03, PNorm = 85.3073, GNorm = 0.1759, lr_0 = 8.5461e-04
Validation prc-auc = 0.270707
Epoch 100
Validation prc-auc = 0.264025
Epoch 101
Validation prc-auc = 0.272788
Epoch 102
Validation prc-auc = 0.274327
Epoch 103
Validation prc-auc = 0.266981
Epoch 104
Loss = 2.8928e-02, PNorm = 85.4931, GNorm = 0.9130, lr_0 = 8.4730e-04
Validation prc-auc = 0.258173
Epoch 105
Validation prc-auc = 0.246047
Epoch 106
Validation prc-auc = 0.245920
Epoch 107
Validation prc-auc = 0.270633
Epoch 108
Validation prc-auc = 0.285974
Epoch 109
Loss = 1.3276e-02, PNorm = 85.6793, GNorm = 0.1830, lr_0 = 8.4006e-04
Validation prc-auc = 0.284292
Epoch 110
Validation prc-auc = 0.291069
Epoch 111
Validation prc-auc = 0.279525
Epoch 112
Validation prc-auc = 0.277428
Epoch 113
Validation prc-auc = 0.279185
Epoch 114
Loss = 1.8808e-02, PNorm = 85.8312, GNorm = 0.0561, lr_0 = 8.3288e-04
Validation prc-auc = 0.276110
Epoch 115
Validation prc-auc = 0.271988
Epoch 116
Validation prc-auc = 0.271654
Epoch 117
Validation prc-auc = 0.270823
Epoch 118
Validation prc-auc = 0.270737
Epoch 119
Loss = 4.4386e-03, PNorm = 85.9360, GNorm = 0.0552, lr_0 = 8.2576e-04
Validation prc-auc = 0.265659
Epoch 120
Validation prc-auc = 0.268518
Epoch 121
Validation prc-auc = 0.263683
Epoch 122
Validation prc-auc = 0.250535
Epoch 123
Validation prc-auc = 0.229241
Epoch 124
Loss = 7.8241e-03, PNorm = 86.0179, GNorm = 0.0496, lr_0 = 8.1870e-04
Validation prc-auc = 0.205385
Epoch 125
Validation prc-auc = 0.239006
Epoch 126
Validation prc-auc = 0.258098
Epoch 127
Validation prc-auc = 0.257373
Epoch 128
Validation prc-auc = 0.263149
Epoch 129
Loss = 2.7940e-02, PNorm = 86.1505, GNorm = 0.4554, lr_0 = 8.1170e-04
Validation prc-auc = 0.271304
Epoch 130
Validation prc-auc = 0.274956
Epoch 131
Validation prc-auc = 0.271552
Epoch 132
Validation prc-auc = 0.268919
Epoch 133
Validation prc-auc = 0.270483
Epoch 134
Loss = 3.8735e-03, PNorm = 86.2603, GNorm = 0.2637, lr_0 = 8.0476e-04
Validation prc-auc = 0.276090
Epoch 135
Validation prc-auc = 0.286859
Epoch 136
Validation prc-auc = 0.281980
Epoch 137
Validation prc-auc = 0.280220
Epoch 138
Validation prc-auc = 0.269243
Epoch 139
Loss = 1.9388e-03, PNorm = 86.3415, GNorm = 0.0418, lr_0 = 7.9788e-04
Validation prc-auc = 0.268408
Epoch 140
Validation prc-auc = 0.256887
Epoch 141
Validation prc-auc = 0.260881
Epoch 142
Validation prc-auc = 0.244569
Epoch 143
Validation prc-auc = 0.241979
Epoch 144
Loss = 1.3388e-03, PNorm = 86.4374, GNorm = 0.0374, lr_0 = 7.9106e-04
Validation prc-auc = 0.231179
Epoch 145
Validation prc-auc = 0.231124
Epoch 146
Validation prc-auc = 0.244250
Epoch 147
Validation prc-auc = 0.286062
Epoch 148
Validation prc-auc = 0.272986
Epoch 149
Loss = 4.9021e-02, PNorm = 86.5866, GNorm = 2.7082, lr_0 = 7.8430e-04
Validation prc-auc = 0.232712
Epoch 150
Validation prc-auc = 0.238059
Epoch 151
Validation prc-auc = 0.248517
Epoch 152
Validation prc-auc = 0.224486
Epoch 153
Validation prc-auc = 0.212868
Epoch 154
Loss = 7.0229e-02, PNorm = 86.9283, GNorm = 0.7214, lr_0 = 7.7759e-04
Validation prc-auc = 0.217212
Epoch 155
Validation prc-auc = 0.272185
Epoch 156
Validation prc-auc = 0.293776
Epoch 157
Validation prc-auc = 0.268372
Epoch 158
Validation prc-auc = 0.227642
Epoch 159
Loss = 1.3424e-02, PNorm = 87.2404, GNorm = 0.0886, lr_0 = 7.7095e-04
Validation prc-auc = 0.232850
Epoch 160
Validation prc-auc = 0.238356
Epoch 161
Validation prc-auc = 0.235740
Epoch 162
Validation prc-auc = 0.236910
Epoch 163
Validation prc-auc = 0.235553
Epoch 164
Loss = 2.3523e-02, PNorm = 87.4460, GNorm = 0.3470, lr_0 = 7.6436e-04
Validation prc-auc = 0.231336
Epoch 165
Validation prc-auc = 0.221248
Epoch 166
Validation prc-auc = 0.219767
Epoch 167
Validation prc-auc = 0.221980
Epoch 168
Validation prc-auc = 0.219601
Epoch 169
Loss = 1.6038e-03, PNorm = 87.6006, GNorm = 0.0283, lr_0 = 7.5782e-04
Validation prc-auc = 0.229195
Epoch 170
Validation prc-auc = 0.241542
Epoch 171
Validation prc-auc = 0.245334
Epoch 172
Validation prc-auc = 0.253989
Epoch 173
Validation prc-auc = 0.266207
Epoch 174
Loss = 5.2102e-03, PNorm = 87.6887, GNorm = 0.0197, lr_0 = 7.5134e-04
Validation prc-auc = 0.264828
Epoch 175
Validation prc-auc = 0.264145
Epoch 176
Validation prc-auc = 0.271491
Epoch 177
Validation prc-auc = 0.276224
Epoch 178
Validation prc-auc = 0.275161
Epoch 179
Loss = 1.3393e-03, PNorm = 87.7670, GNorm = 0.0348, lr_0 = 7.4492e-04
Validation prc-auc = 0.274605
Epoch 180
Validation prc-auc = 0.277417
Epoch 181
Validation prc-auc = 0.294547
Epoch 182
Validation prc-auc = 0.294507
Epoch 183
Validation prc-auc = 0.301067
Epoch 184
Loss = 6.7950e-04, PNorm = 87.8157, GNorm = 0.0139, lr_0 = 7.3855e-04
Validation prc-auc = 0.301259
Epoch 185
Validation prc-auc = 0.303262
Epoch 186
Validation prc-auc = 0.286288
Epoch 187
Validation prc-auc = 0.286300
Epoch 188
Validation prc-auc = 0.295979
Epoch 189
Loss = 9.6690e-04, PNorm = 87.8530, GNorm = 0.0188, lr_0 = 7.3224e-04
Validation prc-auc = 0.301419
Epoch 190
Validation prc-auc = 0.300685
Epoch 191
Validation prc-auc = 0.294702
Epoch 192
Validation prc-auc = 0.282978
Epoch 193
Validation prc-auc = 0.270727
Epoch 194
Loss = 1.2198e-03, PNorm = 87.8885, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.266163
Epoch 195
Validation prc-auc = 0.270245
Epoch 196
Validation prc-auc = 0.270173
Epoch 197
Validation prc-auc = 0.265934
Epoch 198
Validation prc-auc = 0.270732
Epoch 199
Loss = 6.6885e-04, PNorm = 87.9168, GNorm = 0.0120, lr_0 = 7.1977e-04
Validation prc-auc = 0.271415
Epoch 200
Validation prc-auc = 0.288482
Epoch 201
Validation prc-auc = 0.284893
Epoch 202
Validation prc-auc = 0.285060
Epoch 203
Validation prc-auc = 0.307681
Epoch 204
Loss = 8.4359e-05, PNorm = 87.9388, GNorm = 0.0025, lr_0 = 7.1362e-04
Validation prc-auc = 0.314679
Epoch 205
Validation prc-auc = 0.305623
Epoch 206
Validation prc-auc = 0.310165
Epoch 207
Validation prc-auc = 0.310124
Epoch 208
Validation prc-auc = 0.301500
Epoch 209
Loss = 5.2673e-04, PNorm = 87.9587, GNorm = 0.0489, lr_0 = 7.0752e-04
Validation prc-auc = 0.300496
Epoch 210
Validation prc-auc = 0.284767
Epoch 211
Validation prc-auc = 0.279483
Epoch 212
Validation prc-auc = 0.279546
Epoch 213
Validation prc-auc = 0.279826
Epoch 214
Loss = 1.2627e-04, PNorm = 87.9753, GNorm = 0.0048, lr_0 = 7.0147e-04
Validation prc-auc = 0.284899
Epoch 215
Validation prc-auc = 0.289604
Epoch 216
Validation prc-auc = 0.300577
Epoch 217
Validation prc-auc = 0.300990
Epoch 218
Validation prc-auc = 0.301239
Epoch 219
Loss = 2.5233e-04, PNorm = 87.9907, GNorm = 0.0108, lr_0 = 6.9548e-04
Validation prc-auc = 0.301651
Epoch 220
Validation prc-auc = 0.301411
Epoch 221
Validation prc-auc = 0.301202
Epoch 222
Validation prc-auc = 0.301063
Epoch 223
Validation prc-auc = 0.300274
Epoch 224
Loss = 2.9315e-04, PNorm = 88.0053, GNorm = 0.0191, lr_0 = 6.8953e-04
Validation prc-auc = 0.295728
Epoch 225
Validation prc-auc = 0.295912
Epoch 226
Validation prc-auc = 0.291701
Epoch 227
Validation prc-auc = 0.288034
Epoch 228
Validation prc-auc = 0.291969
Epoch 229
Loss = 2.4461e-04, PNorm = 88.0127, GNorm = 0.0141, lr_0 = 6.8364e-04
Validation prc-auc = 0.296665
Epoch 230
Validation prc-auc = 0.306781
Epoch 231
Validation prc-auc = 0.311456
Epoch 232
Validation prc-auc = 0.311396
Epoch 233
Validation prc-auc = 0.311396
Epoch 234
Loss = 1.2914e-04, PNorm = 88.0232, GNorm = 0.0034, lr_0 = 6.7779e-04
Validation prc-auc = 0.303059
Epoch 235
Validation prc-auc = 0.302870
Epoch 236
Validation prc-auc = 0.302869
Epoch 237
Validation prc-auc = 0.302747
Epoch 238
Validation prc-auc = 0.302210
Epoch 239
Loss = 1.2278e-04, PNorm = 88.0353, GNorm = 0.0029, lr_0 = 6.7200e-04
Validation prc-auc = 0.302092
Epoch 240
Validation prc-auc = 0.302092
Epoch 241
Validation prc-auc = 0.297142
Epoch 242
Validation prc-auc = 0.293438
Epoch 243
Validation prc-auc = 0.293578
Epoch 244
Loss = 7.1495e-05, PNorm = 88.0434, GNorm = 0.0039, lr_0 = 6.6625e-04
Validation prc-auc = 0.293645
Epoch 245
Validation prc-auc = 0.297940
Epoch 246
Validation prc-auc = 0.298526
Epoch 247
Validation prc-auc = 0.298917
Epoch 248
Validation prc-auc = 0.302558
Epoch 249
Loss = 1.0654e-04, PNorm = 88.0486, GNorm = 0.0014, lr_0 = 6.6056e-04
Validation prc-auc = 0.313573
Epoch 250
Validation prc-auc = 0.313504
Epoch 251
Validation prc-auc = 0.313366
Epoch 252
Validation prc-auc = 0.313295
Epoch 253
Validation prc-auc = 0.302538
Epoch 254
Loss = 8.0787e-05, PNorm = 88.0568, GNorm = 0.0034, lr_0 = 6.5491e-04
Validation prc-auc = 0.298417
Epoch 255
Validation prc-auc = 0.298276
Epoch 256
Validation prc-auc = 0.297689
Epoch 257
Validation prc-auc = 0.297427
Epoch 258
Validation prc-auc = 0.297499
Epoch 259
Loss = 5.9110e-05, PNorm = 88.0638, GNorm = 0.0015, lr_0 = 6.4931e-04
Validation prc-auc = 0.298014
Epoch 260
Validation prc-auc = 0.331347
Epoch 261
Validation prc-auc = 0.331277
Epoch 262
Validation prc-auc = 0.331151
Epoch 263
Validation prc-auc = 0.331130
Epoch 264
Loss = 9.1253e-05, PNorm = 88.0694, GNorm = 0.0064, lr_0 = 6.4376e-04
Validation prc-auc = 0.335768
Epoch 265
Validation prc-auc = 0.336187
Epoch 266
Validation prc-auc = 0.335446
Epoch 267
Validation prc-auc = 0.340438
Epoch 268
Validation prc-auc = 0.340500
Epoch 269
Loss = 3.4476e-05, PNorm = 88.0747, GNorm = 0.0014, lr_0 = 6.3826e-04
Validation prc-auc = 0.340638
Epoch 270
Validation prc-auc = 0.340264
Epoch 271
Validation prc-auc = 0.336282
Epoch 272
Validation prc-auc = 0.340074
Epoch 273
Validation prc-auc = 0.340485
Epoch 274
Loss = 5.8804e-05, PNorm = 88.0807, GNorm = 0.0022, lr_0 = 6.3280e-04
Validation prc-auc = 0.346581
Epoch 275
Validation prc-auc = 0.346718
Epoch 276
Validation prc-auc = 0.356756
Epoch 277
Validation prc-auc = 0.356595
Epoch 278
Validation prc-auc = 0.356595
Epoch 279
Loss = 2.7580e-05, PNorm = 88.0861, GNorm = 0.0006, lr_0 = 6.2739e-04
Validation prc-auc = 0.356722
Epoch 280
Validation prc-auc = 0.347937
Epoch 281
Validation prc-auc = 0.347190
Epoch 282
Validation prc-auc = 0.346615
Epoch 283
Validation prc-auc = 0.346484
Epoch 284
Loss = 2.2360e-05, PNorm = 88.0910, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.346354
Epoch 285
Validation prc-auc = 0.346354
Epoch 286
Validation prc-auc = 0.340203
Epoch 287
Validation prc-auc = 0.340214
Epoch 288
Validation prc-auc = 0.335581
Epoch 289
Loss = 3.0651e-05, PNorm = 88.0939, GNorm = 0.0010, lr_0 = 6.1671e-04
Validation prc-auc = 0.335678
Epoch 290
Validation prc-auc = 0.335602
Epoch 291
Validation prc-auc = 0.335329
Epoch 292
Validation prc-auc = 0.335211
Epoch 293
Validation prc-auc = 0.335675
Epoch 294
Loss = 3.2700e-05, PNorm = 88.0977, GNorm = 0.0052, lr_0 = 6.1144e-04
Validation prc-auc = 0.335876
Epoch 295
Validation prc-auc = 0.335479
Epoch 296
Validation prc-auc = 0.335553
Epoch 297
Validation prc-auc = 0.335358
Epoch 298
Validation prc-auc = 0.335749
Epoch 299
Loss = 4.0968e-05, PNorm = 88.1015, GNorm = 0.0011, lr_0 = 6.0621e-04
Validation prc-auc = 0.347635
Model 0 best validation prc-auc = 0.486724 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.402364
Ensemble test prc-auc = 0.402364
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.454787
Epoch 1
Validation prc-auc = 0.455183
Epoch 2
Validation prc-auc = 0.457246
Epoch 3
Validation prc-auc = 0.460642
Epoch 4
Loss = 7.0194e-01, PNorm = 78.5504, GNorm = 3.9290, lr_0 = 6.5000e-04
Validation prc-auc = 0.442238
Epoch 5
Validation prc-auc = 0.453212
Epoch 6
Validation prc-auc = 0.457813
Epoch 7
Validation prc-auc = 0.462085
Epoch 8
Validation prc-auc = 0.472303
Epoch 9
Loss = 6.0939e-01, PNorm = 78.7958, GNorm = 9.0791, lr_0 = 9.9743e-04
Validation prc-auc = 0.462185
Epoch 10
Validation prc-auc = 0.334396
Epoch 11
Validation prc-auc = 0.306692
Epoch 12
Validation prc-auc = 0.346881
Epoch 13
Validation prc-auc = 0.310795
Epoch 14
Loss = 5.8294e-01, PNorm = 79.0813, GNorm = 2.1918, lr_0 = 9.8890e-04
Validation prc-auc = 0.269379
Epoch 15
Validation prc-auc = 0.337045
Epoch 16
Validation prc-auc = 0.434443
Epoch 17
Validation prc-auc = 0.441915
Epoch 18
Validation prc-auc = 0.418510
Epoch 19
Loss = 4.2754e-01, PNorm = 79.3729, GNorm = 1.9741, lr_0 = 9.8045e-04
Validation prc-auc = 0.392791
Epoch 20
Validation prc-auc = 0.420089
Epoch 21
Validation prc-auc = 0.403046
Epoch 22
Validation prc-auc = 0.367614
Epoch 23
Validation prc-auc = 0.337152
Epoch 24
Loss = 4.9482e-01, PNorm = 79.6858, GNorm = 6.3413, lr_0 = 9.7207e-04
Validation prc-auc = 0.267790
Epoch 25
Validation prc-auc = 0.271424
Epoch 26
Validation prc-auc = 0.276685
Epoch 27
Validation prc-auc = 0.296748
Epoch 28
Validation prc-auc = 0.338195
Epoch 29
Loss = 3.4506e-01, PNorm = 80.0312, GNorm = 0.9223, lr_0 = 9.6376e-04
Validation prc-auc = 0.416699
Epoch 30
Validation prc-auc = 0.457941
Epoch 31
Validation prc-auc = 0.450469
Epoch 32
Validation prc-auc = 0.270926
Epoch 33
Validation prc-auc = 0.334869
Epoch 34
Loss = 2.5451e-01, PNorm = 80.3966, GNorm = 2.3773, lr_0 = 9.5552e-04
Validation prc-auc = 0.389059
Epoch 35
Validation prc-auc = 0.405813
Epoch 36
Validation prc-auc = 0.377166
Epoch 37
Validation prc-auc = 0.340674
Epoch 38
Validation prc-auc = 0.234162
Epoch 39
Loss = 2.9442e-01, PNorm = 80.7763, GNorm = 2.9394, lr_0 = 9.4735e-04
Validation prc-auc = 0.199282
Epoch 40
Validation prc-auc = 0.199382
Epoch 41
Validation prc-auc = 0.341877
Epoch 42
Validation prc-auc = 0.327700
Epoch 43
Validation prc-auc = 0.348898
Epoch 44
Loss = 1.9134e-01, PNorm = 81.2134, GNorm = 4.3996, lr_0 = 9.3925e-04
Validation prc-auc = 0.336673
Epoch 45
Validation prc-auc = 0.316691
Epoch 46
Validation prc-auc = 0.336910
Epoch 47
Validation prc-auc = 0.262107
Epoch 48
Validation prc-auc = 0.213369
Epoch 49
Loss = 1.0014e-01, PNorm = 81.6307, GNorm = 1.4613, lr_0 = 9.3122e-04
Validation prc-auc = 0.276573
Epoch 50
Validation prc-auc = 0.293355
Epoch 51
Validation prc-auc = 0.294258
Epoch 52
Validation prc-auc = 0.324556
Epoch 53
Validation prc-auc = 0.287274
Epoch 54
Loss = 3.0780e-01, PNorm = 82.0069, GNorm = 3.1248, lr_0 = 9.2326e-04
Validation prc-auc = 0.275376
Epoch 55
Validation prc-auc = 0.299874
Epoch 56
Validation prc-auc = 0.317530
Epoch 57
Validation prc-auc = 0.309956
Epoch 58
Validation prc-auc = 0.262078
Epoch 59
Loss = 1.6468e-01, PNorm = 82.3623, GNorm = 0.7541, lr_0 = 9.1537e-04
Validation prc-auc = 0.258190
Epoch 60
Validation prc-auc = 0.233495
Epoch 61
Validation prc-auc = 0.216412
Epoch 62
Validation prc-auc = 0.240594
Epoch 63
Validation prc-auc = 0.211781
Epoch 64
Loss = 1.1436e-01, PNorm = 82.7665, GNorm = 2.5405, lr_0 = 9.0754e-04
Validation prc-auc = 0.201018
Epoch 65
Validation prc-auc = 0.215336
Epoch 66
Validation prc-auc = 0.193294
Epoch 67
Validation prc-auc = 0.243513
Epoch 68
Validation prc-auc = 0.351066
Epoch 69
Loss = 1.5013e-01, PNorm = 83.1409, GNorm = 0.7780, lr_0 = 8.9978e-04
Validation prc-auc = 0.197157
Epoch 70
Validation prc-auc = 0.267839
Epoch 71
Validation prc-auc = 0.256646
Epoch 72
Validation prc-auc = 0.227305
Epoch 73
Validation prc-auc = 0.221294
Epoch 74
Loss = 8.8397e-02, PNorm = 83.4788, GNorm = 0.3822, lr_0 = 8.9209e-04
Validation prc-auc = 0.243400
Epoch 75
Validation prc-auc = 0.257978
Epoch 76
Validation prc-auc = 0.249578
Epoch 77
Validation prc-auc = 0.253461
Epoch 78
Validation prc-auc = 0.267424
Epoch 79
Loss = 5.1197e-02, PNorm = 83.7954, GNorm = 0.5032, lr_0 = 8.8447e-04
Validation prc-auc = 0.286060
Epoch 80
Validation prc-auc = 0.258376
Epoch 81
Validation prc-auc = 0.256599
Epoch 82
Validation prc-auc = 0.249125
Epoch 83
Validation prc-auc = 0.232321
Epoch 84
Loss = 2.3021e-02, PNorm = 84.0485, GNorm = 0.8325, lr_0 = 8.7691e-04
Validation prc-auc = 0.240687
Epoch 85
Validation prc-auc = 0.241343
Epoch 86
Validation prc-auc = 0.208891
Epoch 87
Validation prc-auc = 0.189567
Epoch 88
Validation prc-auc = 0.199458
Epoch 89
Loss = 3.3585e-02, PNorm = 84.2406, GNorm = 1.3216, lr_0 = 8.6941e-04
Validation prc-auc = 0.227136
Epoch 90
Validation prc-auc = 0.251822
Epoch 91
Validation prc-auc = 0.242586
Epoch 92
Validation prc-auc = 0.225761
Epoch 93
Validation prc-auc = 0.209611
Epoch 94
Loss = 1.7099e-02, PNorm = 84.4713, GNorm = 0.4374, lr_0 = 8.6198e-04
Validation prc-auc = 0.200627
Epoch 95
Validation prc-auc = 0.239579
Epoch 96
Validation prc-auc = 0.252516
Epoch 97
Validation prc-auc = 0.266980
Epoch 98
Validation prc-auc = 0.286916
Epoch 99
Loss = 7.2256e-03, PNorm = 84.6944, GNorm = 0.2185, lr_0 = 8.5461e-04
Validation prc-auc = 0.287583
Epoch 100
Validation prc-auc = 0.251571
Epoch 101
Validation prc-auc = 0.241074
Epoch 102
Validation prc-auc = 0.205227
Epoch 103
Validation prc-auc = 0.171208
Epoch 104
Loss = 5.0183e-02, PNorm = 84.8603, GNorm = 3.1848, lr_0 = 8.4730e-04
Validation prc-auc = 0.210539
Epoch 105
Validation prc-auc = 0.259053
Epoch 106
Validation prc-auc = 0.281646
Epoch 107
Validation prc-auc = 0.276828
Epoch 108
Validation prc-auc = 0.281906
Epoch 109
Loss = 1.3714e-02, PNorm = 85.0971, GNorm = 0.2898, lr_0 = 8.4006e-04
Validation prc-auc = 0.301848
Epoch 110
Validation prc-auc = 0.310100
Epoch 111
Validation prc-auc = 0.299987
Epoch 112
Validation prc-auc = 0.247440
Epoch 113
Validation prc-auc = 0.216580
Epoch 114
Loss = 1.5790e-02, PNorm = 85.3088, GNorm = 0.0343, lr_0 = 8.3288e-04
Validation prc-auc = 0.197170
Epoch 115
Validation prc-auc = 0.193215
Epoch 116
Validation prc-auc = 0.207562
Epoch 117
Validation prc-auc = 0.219012
Epoch 118
Validation prc-auc = 0.227806
Epoch 119
Loss = 7.5661e-03, PNorm = 85.5044, GNorm = 0.2694, lr_0 = 8.2576e-04
Validation prc-auc = 0.224288
Epoch 120
Validation prc-auc = 0.251187
Epoch 121
Validation prc-auc = 0.226028
Epoch 122
Validation prc-auc = 0.220938
Epoch 123
Validation prc-auc = 0.228040
Epoch 124
Loss = 5.3320e-02, PNorm = 85.6556, GNorm = 1.0852, lr_0 = 8.1870e-04
Validation prc-auc = 0.196481
Epoch 125
Validation prc-auc = 0.205610
Epoch 126
Validation prc-auc = 0.236184
Epoch 127
Validation prc-auc = 0.241849
Epoch 128
Validation prc-auc = 0.259201
Epoch 129
Loss = 3.4204e-02, PNorm = 85.8841, GNorm = 1.9203, lr_0 = 8.1170e-04
Validation prc-auc = 0.260198
Epoch 130
Validation prc-auc = 0.256769
Epoch 131
Validation prc-auc = 0.205087
Epoch 132
Validation prc-auc = 0.194703
Epoch 133
Validation prc-auc = 0.222715
Epoch 134
Loss = 1.1991e-02, PNorm = 86.0542, GNorm = 0.4515, lr_0 = 8.0476e-04
Validation prc-auc = 0.240268
Epoch 135
Validation prc-auc = 0.257440
Epoch 136
Validation prc-auc = 0.261329
Epoch 137
Validation prc-auc = 0.262965
Epoch 138
Validation prc-auc = 0.301042
Epoch 139
Loss = 3.7627e-03, PNorm = 86.2992, GNorm = 0.1044, lr_0 = 7.9788e-04
Validation prc-auc = 0.311650
Epoch 140
Validation prc-auc = 0.320001
Epoch 141
Validation prc-auc = 0.306695
Epoch 142
Validation prc-auc = 0.270824
Epoch 143
Validation prc-auc = 0.253500
Epoch 144
Loss = 7.5574e-03, PNorm = 86.4422, GNorm = 0.2978, lr_0 = 7.9106e-04
Validation prc-auc = 0.244863
Epoch 145
Validation prc-auc = 0.241338
Epoch 146
Validation prc-auc = 0.240851
Epoch 147
Validation prc-auc = 0.241389
Epoch 148
Validation prc-auc = 0.257856
Epoch 149
Loss = 7.9625e-03, PNorm = 86.6006, GNorm = 0.6261, lr_0 = 7.8430e-04
Validation prc-auc = 0.252878
Epoch 150
Validation prc-auc = 0.247248
Epoch 151
Validation prc-auc = 0.250543
Epoch 152
Validation prc-auc = 0.210007
Epoch 153
Validation prc-auc = 0.196377
Epoch 154
Loss = 4.1447e-03, PNorm = 86.7082, GNorm = 0.1345, lr_0 = 7.7759e-04
Validation prc-auc = 0.181752
Epoch 155
Validation prc-auc = 0.175607
Epoch 156
Validation prc-auc = 0.172722
Epoch 157
Validation prc-auc = 0.192026
Epoch 158
Validation prc-auc = 0.201716
Epoch 159
Loss = 1.8512e-03, PNorm = 86.7960, GNorm = 0.0608, lr_0 = 7.7095e-04
Validation prc-auc = 0.215760
Epoch 160
Validation prc-auc = 0.221667
Epoch 161
Validation prc-auc = 0.220881
Epoch 162
Validation prc-auc = 0.226931
Epoch 163
Validation prc-auc = 0.226766
Epoch 164
Loss = 1.4032e-03, PNorm = 86.8615, GNorm = 0.0165, lr_0 = 7.6436e-04
Validation prc-auc = 0.230959
Epoch 165
Validation prc-auc = 0.230962
Epoch 166
Validation prc-auc = 0.231031
Epoch 167
Validation prc-auc = 0.230962
Epoch 168
Validation prc-auc = 0.231195
Epoch 169
Loss = 8.6651e-04, PNorm = 86.8980, GNorm = 0.0151, lr_0 = 7.5782e-04
Validation prc-auc = 0.232348
Epoch 170
Validation prc-auc = 0.233524
Epoch 171
Validation prc-auc = 0.233524
Epoch 172
Validation prc-auc = 0.231752
Epoch 173
Validation prc-auc = 0.231915
Epoch 174
Loss = 3.3568e-04, PNorm = 86.9257, GNorm = 0.0100, lr_0 = 7.5134e-04
Validation prc-auc = 0.230779
Epoch 175
Validation prc-auc = 0.230578
Epoch 176
Validation prc-auc = 0.229896
Epoch 177
Validation prc-auc = 0.227245
Epoch 178
Validation prc-auc = 0.231205
Epoch 179
Loss = 1.4119e-03, PNorm = 86.9472, GNorm = 0.1662, lr_0 = 7.4492e-04
Validation prc-auc = 0.233057
Epoch 180
Validation prc-auc = 0.229963
Epoch 181
Validation prc-auc = 0.230312
Epoch 182
Validation prc-auc = 0.232165
Epoch 183
Validation prc-auc = 0.232298
Epoch 184
Loss = 4.3759e-04, PNorm = 86.9597, GNorm = 0.0093, lr_0 = 7.3855e-04
Validation prc-auc = 0.231458
Epoch 185
Validation prc-auc = 0.231291
Epoch 186
Validation prc-auc = 0.231148
Epoch 187
Validation prc-auc = 0.231077
Epoch 188
Validation prc-auc = 0.230981
Epoch 189
Loss = 3.5932e-04, PNorm = 86.9807, GNorm = 0.0078, lr_0 = 7.3224e-04
Validation prc-auc = 0.230504
Epoch 190
Validation prc-auc = 0.226111
Epoch 191
Validation prc-auc = 0.226568
Epoch 192
Validation prc-auc = 0.225758
Epoch 193
Validation prc-auc = 0.225825
Epoch 194
Loss = 3.8743e-04, PNorm = 86.9999, GNorm = 0.0089, lr_0 = 7.2598e-04
Validation prc-auc = 0.225745
Epoch 195
Validation prc-auc = 0.228954
Epoch 196
Validation prc-auc = 0.228890
Epoch 197
Validation prc-auc = 0.230440
Epoch 198
Validation prc-auc = 0.230437
Epoch 199
Loss = 1.7005e-04, PNorm = 87.0120, GNorm = 0.0049, lr_0 = 7.1977e-04
Validation prc-auc = 0.231265
Epoch 200
Validation prc-auc = 0.231446
Epoch 201
Validation prc-auc = 0.231379
Epoch 202
Validation prc-auc = 0.235159
Epoch 203
Validation prc-auc = 0.235159
Epoch 204
Loss = 1.4709e-04, PNorm = 87.0239, GNorm = 0.0044, lr_0 = 7.1362e-04
Validation prc-auc = 0.235159
Epoch 205
Validation prc-auc = 0.238190
Epoch 206
Validation prc-auc = 0.238191
Epoch 207
Validation prc-auc = 0.235292
Epoch 208
Validation prc-auc = 0.235292
Epoch 209
Loss = 2.4106e-04, PNorm = 87.0373, GNorm = 0.0197, lr_0 = 7.0752e-04
Validation prc-auc = 0.235292
Epoch 210
Validation prc-auc = 0.235228
Epoch 211
Validation prc-auc = 0.235247
Epoch 212
Validation prc-auc = 0.243912
Epoch 213
Validation prc-auc = 0.245091
Epoch 214
Loss = 8.8845e-05, PNorm = 87.0463, GNorm = 0.0031, lr_0 = 7.0147e-04
Validation prc-auc = 0.245023
Epoch 215
Validation prc-auc = 0.245093
Epoch 216
Validation prc-auc = 0.245093
Epoch 217
Validation prc-auc = 0.247590
Epoch 218
Validation prc-auc = 0.247634
Epoch 219
Loss = 1.6314e-04, PNorm = 87.0580, GNorm = 0.0022, lr_0 = 6.9548e-04
Validation prc-auc = 0.246717
Epoch 220
Validation prc-auc = 0.246717
Epoch 221
Validation prc-auc = 0.246717
Epoch 222
Validation prc-auc = 0.251977
Epoch 223
Validation prc-auc = 0.251886
Epoch 224
Loss = 5.7891e-04, PNorm = 87.0689, GNorm = 0.0141, lr_0 = 6.8953e-04
Validation prc-auc = 0.252070
Epoch 225
Validation prc-auc = 0.252165
Epoch 226
Validation prc-auc = 0.252196
Epoch 227
Validation prc-auc = 0.252196
Epoch 228
Validation prc-auc = 0.252120
Epoch 229
Loss = 9.1246e-05, PNorm = 87.0767, GNorm = 0.0028, lr_0 = 6.8364e-04
Validation prc-auc = 0.252186
Epoch 230
Validation prc-auc = 0.252196
Epoch 231
Validation prc-auc = 0.252196
Epoch 232
Validation prc-auc = 0.252196
Epoch 233
Validation prc-auc = 0.259637
Epoch 234
Loss = 1.0015e-04, PNorm = 87.0887, GNorm = 0.0019, lr_0 = 6.7779e-04
Validation prc-auc = 0.259478
Epoch 235
Validation prc-auc = 0.259461
Epoch 236
Validation prc-auc = 0.259640
Epoch 237
Validation prc-auc = 0.259782
Epoch 238
Validation prc-auc = 0.260681
Epoch 239
Loss = 6.6957e-05, PNorm = 87.0977, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.260835
Epoch 240
Validation prc-auc = 0.260910
Epoch 241
Validation prc-auc = 0.264819
Epoch 242
Validation prc-auc = 0.265838
Epoch 243
Validation prc-auc = 0.265932
Epoch 244
Loss = 5.9781e-05, PNorm = 87.1072, GNorm = 0.0013, lr_0 = 6.6625e-04
Validation prc-auc = 0.265996
Epoch 245
Validation prc-auc = 0.265902
Epoch 246
Validation prc-auc = 0.264921
Epoch 247
Validation prc-auc = 0.264822
Epoch 248
Validation prc-auc = 0.264643
Epoch 249
Loss = 4.1430e-05, PNorm = 87.1184, GNorm = 0.0010, lr_0 = 6.6056e-04
Validation prc-auc = 0.263816
Epoch 250
Validation prc-auc = 0.264666
Epoch 251
Validation prc-auc = 0.263755
Epoch 252
Validation prc-auc = 0.263099
Epoch 253
Validation prc-auc = 0.263928
Epoch 254
Loss = 4.1046e-05, PNorm = 87.1295, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.264008
Epoch 255
Validation prc-auc = 0.265221
Epoch 256
Validation prc-auc = 0.266494
Epoch 257
Validation prc-auc = 0.266876
Epoch 258
Validation prc-auc = 0.266876
Epoch 259
Loss = 4.6054e-05, PNorm = 87.1382, GNorm = 0.0006, lr_0 = 6.4931e-04
Validation prc-auc = 0.265720
Epoch 260
Validation prc-auc = 0.264760
Epoch 261
Validation prc-auc = 0.265828
Epoch 262
Validation prc-auc = 0.267399
Epoch 263
Validation prc-auc = 0.268491
Epoch 264
Loss = 2.0171e-05, PNorm = 87.1470, GNorm = 0.0006, lr_0 = 6.4376e-04
Validation prc-auc = 0.268647
Epoch 265
Validation prc-auc = 0.268741
Epoch 266
Validation prc-auc = 0.267219
Epoch 267
Validation prc-auc = 0.268426
Epoch 268
Validation prc-auc = 0.263798
Epoch 269
Loss = 1.1883e-05, PNorm = 87.1543, GNorm = 0.0009, lr_0 = 6.3826e-04
Validation prc-auc = 0.262209
Epoch 270
Validation prc-auc = 0.263728
Epoch 271
Validation prc-auc = 0.267963
Epoch 272
Validation prc-auc = 0.276863
Epoch 273
Validation prc-auc = 0.281979
Epoch 274
Loss = 1.8071e-05, PNorm = 87.1597, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.282208
Epoch 275
Validation prc-auc = 0.273036
Epoch 276
Validation prc-auc = 0.271376
Epoch 277
Validation prc-auc = 0.270654
Epoch 278
Validation prc-auc = 0.267815
Epoch 279
Loss = 1.2040e-05, PNorm = 87.1653, GNorm = 0.0022, lr_0 = 6.2739e-04
Validation prc-auc = 0.254367
Epoch 280
Validation prc-auc = 0.248043
Epoch 281
Validation prc-auc = 0.254376
Epoch 282
Validation prc-auc = 0.255854
Epoch 283
Validation prc-auc = 0.257660
Epoch 284
Loss = 1.3937e-05, PNorm = 87.1688, GNorm = 0.0005, lr_0 = 6.2203e-04
Validation prc-auc = 0.261545
Epoch 285
Validation prc-auc = 0.264887
Epoch 286
Validation prc-auc = 0.264821
Epoch 287
Validation prc-auc = 0.264757
Epoch 288
Validation prc-auc = 0.262994
Epoch 289
Loss = 7.4108e-06, PNorm = 87.1729, GNorm = 0.0005, lr_0 = 6.1671e-04
Validation prc-auc = 0.263275
Epoch 290
Validation prc-auc = 0.262037
Epoch 291
Validation prc-auc = 0.262037
Epoch 292
Validation prc-auc = 0.261647
Epoch 293
Validation prc-auc = 0.261562
Epoch 294
Loss = 8.6336e-06, PNorm = 87.1768, GNorm = 0.0006, lr_0 = 6.1144e-04
Validation prc-auc = 0.257813
Epoch 295
Validation prc-auc = 0.280159
Epoch 296
Validation prc-auc = 0.282587
Epoch 297
Validation prc-auc = 0.285804
Epoch 298
Validation prc-auc = 0.282537
Epoch 299
Loss = 2.0865e-05, PNorm = 87.1799, GNorm = 0.0019, lr_0 = 6.0621e-04
Validation prc-auc = 0.281201
Model 0 best validation prc-auc = 0.472303 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387643
Ensemble test prc-auc = 0.387643
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.434076
Epoch 1
Validation prc-auc = 0.457420
Epoch 2
Validation prc-auc = 0.458604
Epoch 3
Validation prc-auc = 0.471728
Epoch 4
Loss = 6.6940e-01, PNorm = 78.5476, GNorm = 6.7897, lr_0 = 6.5000e-04
Validation prc-auc = 0.459812
Epoch 5
Validation prc-auc = 0.441032
Epoch 6
Validation prc-auc = 0.481371
Epoch 7
Validation prc-auc = 0.465603
Epoch 8
Validation prc-auc = 0.490614
Epoch 9
Loss = 7.5357e-01, PNorm = 78.7818, GNorm = 4.4542, lr_0 = 9.9743e-04
Validation prc-auc = 0.478565
Epoch 10
Validation prc-auc = 0.321278
Epoch 11
Validation prc-auc = 0.329693
Epoch 12
Validation prc-auc = 0.402455
Epoch 13
Validation prc-auc = 0.457976
Epoch 14
Loss = 4.9512e-01, PNorm = 79.0603, GNorm = 4.6236, lr_0 = 9.8890e-04
Validation prc-auc = 0.465991
Epoch 15
Validation prc-auc = 0.474146
Epoch 16
Validation prc-auc = 0.494079
Epoch 17
Validation prc-auc = 0.481656
Epoch 18
Validation prc-auc = 0.491480
Epoch 19
Loss = 6.0016e-01, PNorm = 79.3179, GNorm = 11.3375, lr_0 = 9.8045e-04
Validation prc-auc = 0.356324
Epoch 20
Validation prc-auc = 0.319028
Epoch 21
Validation prc-auc = 0.396666
Epoch 22
Validation prc-auc = 0.327262
Epoch 23
Validation prc-auc = 0.249020
Epoch 24
Loss = 5.2568e-01, PNorm = 79.5741, GNorm = 3.9257, lr_0 = 9.7207e-04
Validation prc-auc = 0.344926
Epoch 25
Validation prc-auc = 0.463539
Epoch 26
Validation prc-auc = 0.462075
Epoch 27
Validation prc-auc = 0.463508
Epoch 28
Validation prc-auc = 0.462585
Epoch 29
Loss = 4.8583e-01, PNorm = 79.8787, GNorm = 1.1889, lr_0 = 9.6376e-04
Validation prc-auc = 0.441641
Epoch 30
Validation prc-auc = 0.350212
Epoch 31
Validation prc-auc = 0.307020
Epoch 32
Validation prc-auc = 0.295028
Epoch 33
Validation prc-auc = 0.308890
Epoch 34
Loss = 3.5980e-01, PNorm = 80.1620, GNorm = 1.2794, lr_0 = 9.5552e-04
Validation prc-auc = 0.347034
Epoch 35
Validation prc-auc = 0.345844
Epoch 36
Validation prc-auc = 0.329048
Epoch 37
Validation prc-auc = 0.338035
Epoch 38
Validation prc-auc = 0.385873
Epoch 39
Loss = 3.6370e-01, PNorm = 80.4752, GNorm = 3.3556, lr_0 = 9.4735e-04
Validation prc-auc = 0.354778
Epoch 40
Validation prc-auc = 0.310202
Epoch 41
Validation prc-auc = 0.297669
Epoch 42
Validation prc-auc = 0.324231
Epoch 43
Validation prc-auc = 0.296734
Epoch 44
Loss = 3.6850e-01, PNorm = 80.7925, GNorm = 5.2399, lr_0 = 9.3925e-04
Validation prc-auc = 0.279638
Epoch 45
Validation prc-auc = 0.334626
Epoch 46
Validation prc-auc = 0.341671
Epoch 47
Validation prc-auc = 0.358256
Epoch 48
Validation prc-auc = 0.330206
Epoch 49
Loss = 2.1353e-01, PNorm = 81.1628, GNorm = 2.3248, lr_0 = 9.3122e-04
Validation prc-auc = 0.352046
Epoch 50
Validation prc-auc = 0.303014
Epoch 51
Validation prc-auc = 0.310926
Epoch 52
Validation prc-auc = 0.307721
Epoch 53
Validation prc-auc = 0.320798
Epoch 54
Loss = 2.5593e-01, PNorm = 81.5512, GNorm = 5.2760, lr_0 = 9.2326e-04
Validation prc-auc = 0.343211
Epoch 55
Validation prc-auc = 0.337257
Epoch 56
Validation prc-auc = 0.359772
Epoch 57
Validation prc-auc = 0.377172
Epoch 58
Validation prc-auc = 0.342004
Epoch 59
Loss = 1.8163e-01, PNorm = 81.9307, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation prc-auc = 0.298545
Epoch 60
Validation prc-auc = 0.280353
Epoch 61
Validation prc-auc = 0.294281
Epoch 62
Validation prc-auc = 0.265543
Epoch 63
Validation prc-auc = 0.258436
Epoch 64
Loss = 1.3860e-01, PNorm = 82.3406, GNorm = 1.8254, lr_0 = 9.0754e-04
Validation prc-auc = 0.274408
Epoch 65
Validation prc-auc = 0.191392
Epoch 66
Validation prc-auc = 0.298554
Epoch 67
Validation prc-auc = 0.304117
Epoch 68
Validation prc-auc = 0.362464
Epoch 69
Loss = 6.8738e-02, PNorm = 82.7452, GNorm = 1.6550, lr_0 = 8.9978e-04
Validation prc-auc = 0.373048
Epoch 70
Validation prc-auc = 0.370687
Epoch 71
Validation prc-auc = 0.368884
Epoch 72
Validation prc-auc = 0.334850
Epoch 73
Validation prc-auc = 0.307704
Epoch 74
Loss = 6.8877e-02, PNorm = 83.0944, GNorm = 0.3143, lr_0 = 8.9209e-04
Validation prc-auc = 0.269460
Epoch 75
Validation prc-auc = 0.229206
Epoch 76
Validation prc-auc = 0.269542
Epoch 77
Validation prc-auc = 0.207595
Epoch 78
Validation prc-auc = 0.290756
Epoch 79
Loss = 8.6926e-02, PNorm = 83.4015, GNorm = 0.5161, lr_0 = 8.8447e-04
Validation prc-auc = 0.283063
Epoch 80
Validation prc-auc = 0.283195
Epoch 81
Validation prc-auc = 0.328266
Epoch 82
Validation prc-auc = 0.342782
Epoch 83
Validation prc-auc = 0.306091
Epoch 84
Loss = 2.3302e-01, PNorm = 83.7335, GNorm = 1.7786, lr_0 = 8.7691e-04
Validation prc-auc = 0.279411
Epoch 85
Validation prc-auc = 0.281117
Epoch 86
Validation prc-auc = 0.246712
Epoch 87
Validation prc-auc = 0.224165
Epoch 88
Validation prc-auc = 0.287092
Epoch 89
Loss = 4.5509e-02, PNorm = 84.1042, GNorm = 0.6834, lr_0 = 8.6941e-04
Validation prc-auc = 0.308344
Epoch 90
Validation prc-auc = 0.328625
Epoch 91
Validation prc-auc = 0.309906
Epoch 92
Validation prc-auc = 0.261949
Epoch 93
Validation prc-auc = 0.238334
Epoch 94
Loss = 2.9086e-02, PNorm = 84.3840, GNorm = 0.7699, lr_0 = 8.6198e-04
Validation prc-auc = 0.233986
Epoch 95
Validation prc-auc = 0.242076
Epoch 96
Validation prc-auc = 0.234171
Epoch 97
Validation prc-auc = 0.259769
Epoch 98
Validation prc-auc = 0.260372
Epoch 99
Loss = 2.7894e-02, PNorm = 84.6258, GNorm = 0.5002, lr_0 = 8.5461e-04
Validation prc-auc = 0.241337
Epoch 100
Validation prc-auc = 0.230159
Epoch 101
Validation prc-auc = 0.240429
Epoch 102
Validation prc-auc = 0.245848
Epoch 103
Validation prc-auc = 0.256310
Epoch 104
Loss = 4.0704e-02, PNorm = 84.8366, GNorm = 1.1440, lr_0 = 8.4730e-04
Validation prc-auc = 0.229493
Epoch 105
Validation prc-auc = 0.226121
Epoch 106
Validation prc-auc = 0.222959
Epoch 107
Validation prc-auc = 0.226675
Epoch 108
Validation prc-auc = 0.242013
Epoch 109
Loss = 1.4584e-02, PNorm = 84.9962, GNorm = 0.6123, lr_0 = 8.4006e-04
Validation prc-auc = 0.256096
Epoch 110
Validation prc-auc = 0.262724
Epoch 111
Validation prc-auc = 0.237542
Epoch 112
Validation prc-auc = 0.212855
Epoch 113
Validation prc-auc = 0.222727
Epoch 114
Loss = 2.4388e-02, PNorm = 85.1564, GNorm = 0.9167, lr_0 = 8.3288e-04
Validation prc-auc = 0.242274
Epoch 115
Validation prc-auc = 0.279204
Epoch 116
Validation prc-auc = 0.278679
Epoch 117
Validation prc-auc = 0.253609
Epoch 118
Validation prc-auc = 0.214742
Epoch 119
Loss = 3.5193e-02, PNorm = 85.3554, GNorm = 0.8935, lr_0 = 8.2576e-04
Validation prc-auc = 0.189706
Epoch 120
Validation prc-auc = 0.186311
Epoch 121
Validation prc-auc = 0.245165
Epoch 122
Validation prc-auc = 0.237705
Epoch 123
Validation prc-auc = 0.240659
Epoch 124
Loss = 2.4166e-02, PNorm = 85.5488, GNorm = 0.2369, lr_0 = 8.1870e-04
Validation prc-auc = 0.250475
Epoch 125
Validation prc-auc = 0.266610
Epoch 126
Validation prc-auc = 0.266784
Epoch 127
Validation prc-auc = 0.270365
Epoch 128
Validation prc-auc = 0.245779
Epoch 129
Loss = 7.9705e-03, PNorm = 85.7155, GNorm = 0.1555, lr_0 = 8.1170e-04
Validation prc-auc = 0.231842
Epoch 130
Validation prc-auc = 0.214591
Epoch 131
Validation prc-auc = 0.212922
Epoch 132
Validation prc-auc = 0.210431
Epoch 133
Validation prc-auc = 0.209643
Epoch 134
Loss = 3.5030e-03, PNorm = 85.8338, GNorm = 0.2374, lr_0 = 8.0476e-04
Validation prc-auc = 0.207306
Epoch 135
Validation prc-auc = 0.200479
Epoch 136
Validation prc-auc = 0.202470
Epoch 137
Validation prc-auc = 0.205772
Epoch 138
Validation prc-auc = 0.209165
Epoch 139
Loss = 3.8180e-03, PNorm = 85.9040, GNorm = 0.2716, lr_0 = 7.9788e-04
Validation prc-auc = 0.213847
Epoch 140
Validation prc-auc = 0.218795
Epoch 141
Validation prc-auc = 0.210443
Epoch 142
Validation prc-auc = 0.219385
Epoch 143
Validation prc-auc = 0.244349
Epoch 144
Loss = 1.2857e-02, PNorm = 85.9696, GNorm = 0.3995, lr_0 = 7.9106e-04
Validation prc-auc = 0.237443
Epoch 145
Validation prc-auc = 0.246441
Epoch 146
Validation prc-auc = 0.253707
Epoch 147
Validation prc-auc = 0.269453
Epoch 148
Validation prc-auc = 0.283431
Epoch 149
Loss = 8.9761e-03, PNorm = 86.1367, GNorm = 0.2976, lr_0 = 7.8430e-04
Validation prc-auc = 0.289997
Epoch 150
Validation prc-auc = 0.295357
Epoch 151
Validation prc-auc = 0.261798
Epoch 152
Validation prc-auc = 0.235727
Epoch 153
Validation prc-auc = 0.247289
Epoch 154
Loss = 1.5452e-02, PNorm = 86.2802, GNorm = 0.1277, lr_0 = 7.7759e-04
Validation prc-auc = 0.241072
Epoch 155
Validation prc-auc = 0.242246
Epoch 156
Validation prc-auc = 0.214299
Epoch 157
Validation prc-auc = 0.199380
Epoch 158
Validation prc-auc = 0.239951
Epoch 159
Loss = 5.9379e-03, PNorm = 86.4244, GNorm = 0.1018, lr_0 = 7.7095e-04
Validation prc-auc = 0.246812
Epoch 160
Validation prc-auc = 0.255179
Epoch 161
Validation prc-auc = 0.248417
Epoch 162
Validation prc-auc = 0.238683
Epoch 163
Validation prc-auc = 0.237195
Epoch 164
Loss = 1.9421e-03, PNorm = 86.5171, GNorm = 0.0091, lr_0 = 7.6436e-04
Validation prc-auc = 0.250266
Epoch 165
Validation prc-auc = 0.252963
Epoch 166
Validation prc-auc = 0.254370
Epoch 167
Validation prc-auc = 0.250415
Epoch 168
Validation prc-auc = 0.249695
Epoch 169
Loss = 1.3199e-03, PNorm = 86.5762, GNorm = 0.0447, lr_0 = 7.5782e-04
Validation prc-auc = 0.249349
Epoch 170
Validation prc-auc = 0.246538
Epoch 171
Validation prc-auc = 0.240236
Epoch 172
Validation prc-auc = 0.240534
Epoch 173
Validation prc-auc = 0.235577
Epoch 174
Loss = 2.2461e-03, PNorm = 86.6135, GNorm = 0.2446, lr_0 = 7.5134e-04
Validation prc-auc = 0.244731
Epoch 175
Validation prc-auc = 0.246985
Epoch 176
Validation prc-auc = 0.249017
Epoch 177
Validation prc-auc = 0.252572
Epoch 178
Validation prc-auc = 0.251968
Epoch 179
Loss = 6.3653e-04, PNorm = 86.6483, GNorm = 0.0177, lr_0 = 7.4492e-04
Validation prc-auc = 0.249646
Epoch 180
Validation prc-auc = 0.242470
Epoch 181
Validation prc-auc = 0.243353
Epoch 182
Validation prc-auc = 0.243127
Epoch 183
Validation prc-auc = 0.240641
Epoch 184
Loss = 4.6618e-04, PNorm = 86.6758, GNorm = 0.0062, lr_0 = 7.3855e-04
Validation prc-auc = 0.237657
Epoch 185
Validation prc-auc = 0.237527
Epoch 186
Validation prc-auc = 0.239330
Epoch 187
Validation prc-auc = 0.238970
Epoch 188
Validation prc-auc = 0.243761
Epoch 189
Loss = 1.4330e-03, PNorm = 86.6921, GNorm = 0.0315, lr_0 = 7.3224e-04
Validation prc-auc = 0.249133
Epoch 190
Validation prc-auc = 0.253101
Epoch 191
Validation prc-auc = 0.245300
Epoch 192
Validation prc-auc = 0.252355
Epoch 193
Validation prc-auc = 0.253956
Epoch 194
Loss = 3.3508e-04, PNorm = 86.7066, GNorm = 0.0125, lr_0 = 7.2598e-04
Validation prc-auc = 0.253320
Epoch 195
Validation prc-auc = 0.253719
Epoch 196
Validation prc-auc = 0.254532
Epoch 197
Validation prc-auc = 0.254145
Epoch 198
Validation prc-auc = 0.254142
Epoch 199
Loss = 3.3925e-04, PNorm = 86.7180, GNorm = 0.0175, lr_0 = 7.1977e-04
Validation prc-auc = 0.254220
Epoch 200
Validation prc-auc = 0.254019
Epoch 201
Validation prc-auc = 0.254220
Epoch 202
Validation prc-auc = 0.254220
Epoch 203
Validation prc-auc = 0.253649
Epoch 204
Loss = 1.5806e-04, PNorm = 86.7262, GNorm = 0.0059, lr_0 = 7.1362e-04
Validation prc-auc = 0.252956
Epoch 205
Validation prc-auc = 0.252673
Epoch 206
Validation prc-auc = 0.252604
Epoch 207
Validation prc-auc = 0.252604
Epoch 208
Validation prc-auc = 0.252525
Epoch 209
Loss = 8.6087e-05, PNorm = 86.7315, GNorm = 0.0030, lr_0 = 7.0752e-04
Validation prc-auc = 0.250358
Epoch 210
Validation prc-auc = 0.253926
Epoch 211
Validation prc-auc = 0.253731
Epoch 212
Validation prc-auc = 0.254005
Epoch 213
Validation prc-auc = 0.254205
Epoch 214
Loss = 2.0440e-04, PNorm = 86.7374, GNorm = 0.0069, lr_0 = 7.0147e-04
Validation prc-auc = 0.254903
Epoch 215
Validation prc-auc = 0.257509
Epoch 216
Validation prc-auc = 0.257265
Epoch 217
Validation prc-auc = 0.257896
Epoch 218
Validation prc-auc = 0.257896
Epoch 219
Loss = 1.3274e-04, PNorm = 86.7445, GNorm = 0.0034, lr_0 = 6.9548e-04
Validation prc-auc = 0.257900
Epoch 220
Validation prc-auc = 0.252493
Epoch 221
Validation prc-auc = 0.252493
Epoch 222
Validation prc-auc = 0.252430
Epoch 223
Validation prc-auc = 0.252665
Epoch 224
Loss = 2.3222e-04, PNorm = 86.7508, GNorm = 0.0105, lr_0 = 6.8953e-04
Validation prc-auc = 0.251845
Epoch 225
Validation prc-auc = 0.251845
Epoch 226
Validation prc-auc = 0.251988
Epoch 227
Validation prc-auc = 0.256996
Epoch 228
Validation prc-auc = 0.256713
Epoch 229
Loss = 1.0546e-04, PNorm = 86.7573, GNorm = 0.0036, lr_0 = 6.8364e-04
Validation prc-auc = 0.256713
Epoch 230
Validation prc-auc = 0.256467
Epoch 231
Validation prc-auc = 0.256467
Epoch 232
Validation prc-auc = 0.256197
Epoch 233
Validation prc-auc = 0.255886
Epoch 234
Loss = 1.0012e-04, PNorm = 86.7625, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.253234
Epoch 235
Validation prc-auc = 0.250456
Epoch 236
Validation prc-auc = 0.249570
Epoch 237
Validation prc-auc = 0.249646
Epoch 238
Validation prc-auc = 0.249646
Epoch 239
Loss = 1.4534e-04, PNorm = 86.7672, GNorm = 0.0069, lr_0 = 6.7200e-04
Validation prc-auc = 0.249527
Epoch 240
Validation prc-auc = 0.248775
Epoch 241
Validation prc-auc = 0.250517
Epoch 242
Validation prc-auc = 0.250440
Epoch 243
Validation prc-auc = 0.250516
Epoch 244
Loss = 1.7778e-04, PNorm = 86.7717, GNorm = 0.0076, lr_0 = 6.6625e-04
Validation prc-auc = 0.250440
Epoch 245
Validation prc-auc = 0.249983
Epoch 246
Validation prc-auc = 0.249986
Epoch 247
Validation prc-auc = 0.249986
Epoch 248
Validation prc-auc = 0.250038
Epoch 249
Loss = 5.5124e-05, PNorm = 86.7762, GNorm = 0.0019, lr_0 = 6.6056e-04
Validation prc-auc = 0.250105
Epoch 250
Validation prc-auc = 0.249858
Epoch 251
Validation prc-auc = 0.249911
Epoch 252
Validation prc-auc = 0.249723
Epoch 253
Validation prc-auc = 0.250603
Epoch 254
Loss = 6.4436e-05, PNorm = 86.7802, GNorm = 0.0013, lr_0 = 6.5491e-04
Validation prc-auc = 0.249911
Epoch 255
Validation prc-auc = 0.249990
Epoch 256
Validation prc-auc = 0.249803
Epoch 257
Validation prc-auc = 0.249567
Epoch 258
Validation prc-auc = 0.249567
Epoch 259
Loss = 9.6219e-05, PNorm = 86.7837, GNorm = 0.0085, lr_0 = 6.4931e-04
Validation prc-auc = 0.249567
Epoch 260
Validation prc-auc = 0.249635
Epoch 261
Validation prc-auc = 0.249822
Epoch 262
Validation prc-auc = 0.250218
Epoch 263
Validation prc-auc = 0.250218
Epoch 264
Loss = 5.8861e-05, PNorm = 86.7876, GNorm = 0.0022, lr_0 = 6.4376e-04
Validation prc-auc = 0.250150
Epoch 265
Validation prc-auc = 0.250150
Epoch 266
Validation prc-auc = 0.249923
Epoch 267
Validation prc-auc = 0.256431
Epoch 268
Validation prc-auc = 0.256215
Epoch 269
Loss = 1.0507e-04, PNorm = 86.7916, GNorm = 0.0065, lr_0 = 6.3826e-04
Validation prc-auc = 0.256013
Epoch 270
Validation prc-auc = 0.256090
Epoch 271
Validation prc-auc = 0.256090
Epoch 272
Validation prc-auc = 0.256090
Epoch 273
Validation prc-auc = 0.256090
Epoch 274
Loss = 5.5034e-05, PNorm = 86.7959, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.256090
Epoch 275
Validation prc-auc = 0.255896
Epoch 276
Validation prc-auc = 0.255819
Epoch 277
Validation prc-auc = 0.255611
Epoch 278
Validation prc-auc = 0.255611
Epoch 279
Loss = 7.1451e-05, PNorm = 86.8009, GNorm = 0.0042, lr_0 = 6.2739e-04
Validation prc-auc = 0.255611
Epoch 280
Validation prc-auc = 0.255597
Epoch 281
Validation prc-auc = 0.255597
Epoch 282
Validation prc-auc = 0.255397
Epoch 283
Validation prc-auc = 0.254757
Epoch 284
Loss = 1.0247e-04, PNorm = 86.8062, GNorm = 0.0051, lr_0 = 6.2203e-04
Validation prc-auc = 0.254957
Epoch 285
Validation prc-auc = 0.255151
Epoch 286
Validation prc-auc = 0.255353
Epoch 287
Validation prc-auc = 0.255629
Epoch 288
Validation prc-auc = 0.255629
Epoch 289
Loss = 5.7148e-05, PNorm = 86.8121, GNorm = 0.0029, lr_0 = 6.1671e-04
Validation prc-auc = 0.255706
Epoch 290
Validation prc-auc = 0.256346
Epoch 291
Validation prc-auc = 0.256346
Epoch 292
Validation prc-auc = 0.256063
Epoch 293
Validation prc-auc = 0.255600
Epoch 294
Loss = 5.4535e-05, PNorm = 86.8190, GNorm = 0.0005, lr_0 = 6.1144e-04
Validation prc-auc = 0.255335
Epoch 295
Validation prc-auc = 0.255335
Epoch 296
Validation prc-auc = 0.255335
Epoch 297
Validation prc-auc = 0.255143
Epoch 298
Validation prc-auc = 0.255143
Epoch 299
Loss = 4.1452e-05, PNorm = 86.8257, GNorm = 0.0015, lr_0 = 6.0621e-04
Validation prc-auc = 0.255216
Model 0 best validation prc-auc = 0.494079 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.473718
Ensemble test prc-auc = 0.473718
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.444433
Epoch 1
Validation prc-auc = 0.456129
Epoch 2
Validation prc-auc = 0.420647
Epoch 3
Validation prc-auc = 0.440192
Epoch 4
Loss = 6.5323e-01, PNorm = 78.5536, GNorm = 3.8553, lr_0 = 6.5000e-04
Validation prc-auc = 0.469897
Epoch 5
Validation prc-auc = 0.441008
Epoch 6
Validation prc-auc = 0.439744
Epoch 7
Validation prc-auc = 0.429229
Epoch 8
Validation prc-auc = 0.430780
Epoch 9
Loss = 6.5283e-01, PNorm = 78.7980, GNorm = 8.9798, lr_0 = 9.9743e-04
Validation prc-auc = 0.299335
Epoch 10
Validation prc-auc = 0.310533
Epoch 11
Validation prc-auc = 0.376633
Epoch 12
Validation prc-auc = 0.478044
Epoch 13
Validation prc-auc = 0.421652
Epoch 14
Loss = 5.3995e-01, PNorm = 79.1048, GNorm = 0.7729, lr_0 = 9.8890e-04
Validation prc-auc = 0.412349
Epoch 15
Validation prc-auc = 0.403937
Epoch 16
Validation prc-auc = 0.487608
Epoch 17
Validation prc-auc = 0.466166
Epoch 18
Validation prc-auc = 0.297784
Epoch 19
Loss = 4.1483e-01, PNorm = 79.4444, GNorm = 3.4478, lr_0 = 9.8045e-04
Validation prc-auc = 0.271431
Epoch 20
Validation prc-auc = 0.322065
Epoch 21
Validation prc-auc = 0.319643
Epoch 22
Validation prc-auc = 0.320843
Epoch 23
Validation prc-auc = 0.268977
Epoch 24
Loss = 5.0285e-01, PNorm = 79.8039, GNorm = 4.1674, lr_0 = 9.7207e-04
Validation prc-auc = 0.289349
Epoch 25
Validation prc-auc = 0.392423
Epoch 26
Validation prc-auc = 0.275621
Epoch 27
Validation prc-auc = 0.306595
Epoch 28
Validation prc-auc = 0.353579
Epoch 29
Loss = 4.1142e-01, PNorm = 80.2021, GNorm = 2.4667, lr_0 = 9.6376e-04
Validation prc-auc = 0.349656
Epoch 30
Validation prc-auc = 0.418446
Epoch 31
Validation prc-auc = 0.441640
Epoch 32
Validation prc-auc = 0.361011
Epoch 33
Validation prc-auc = 0.355823
Epoch 34
Loss = 3.6353e-01, PNorm = 80.5821, GNorm = 7.7298, lr_0 = 9.5552e-04
Validation prc-auc = 0.330606
Epoch 35
Validation prc-auc = 0.365648
Epoch 36
Validation prc-auc = 0.336102
Epoch 37
Validation prc-auc = 0.361568
Epoch 38
Validation prc-auc = 0.382436
Epoch 39
Loss = 2.5884e-01, PNorm = 81.0056, GNorm = 2.0205, lr_0 = 9.4735e-04
Validation prc-auc = 0.399115
Epoch 40
Validation prc-auc = 0.387232
Epoch 41
Validation prc-auc = 0.351931
Epoch 42
Validation prc-auc = 0.235007
Epoch 43
Validation prc-auc = 0.243459
Epoch 44
Loss = 1.6512e-01, PNorm = 81.4481, GNorm = 1.2873, lr_0 = 9.3925e-04
Validation prc-auc = 0.226750
Epoch 45
Validation prc-auc = 0.359088
Epoch 46
Validation prc-auc = 0.358515
Epoch 47
Validation prc-auc = 0.339205
Epoch 48
Validation prc-auc = 0.335852
Epoch 49
Loss = 1.2805e-01, PNorm = 81.9343, GNorm = 1.6719, lr_0 = 9.3122e-04
Validation prc-auc = 0.320263
Epoch 50
Validation prc-auc = 0.314936
Epoch 51
Validation prc-auc = 0.317689
Epoch 52
Validation prc-auc = 0.315497
Epoch 53
Validation prc-auc = 0.296828
Epoch 54
Loss = 8.5127e-02, PNorm = 82.3601, GNorm = 0.4456, lr_0 = 9.2326e-04
Validation prc-auc = 0.182812
Epoch 55
Validation prc-auc = 0.227757
Epoch 56
Validation prc-auc = 0.256377
Epoch 57
Validation prc-auc = 0.194120
Epoch 58
Validation prc-auc = 0.242637
Epoch 59
Loss = 7.3885e-02, PNorm = 82.7524, GNorm = 1.2944, lr_0 = 9.1537e-04
Validation prc-auc = 0.301696
Epoch 60
Validation prc-auc = 0.279037
Epoch 61
Validation prc-auc = 0.307725
Epoch 62
Validation prc-auc = 0.323613
Epoch 63
Validation prc-auc = 0.291489
Epoch 64
Loss = 4.3935e-02, PNorm = 83.1453, GNorm = 0.3841, lr_0 = 9.0754e-04
Validation prc-auc = 0.237511
Epoch 65
Validation prc-auc = 0.206105
Epoch 66
Validation prc-auc = 0.213431
Epoch 67
Validation prc-auc = 0.201107
Epoch 68
Validation prc-auc = 0.205966
Epoch 69
Loss = 3.8325e-02, PNorm = 83.5038, GNorm = 0.6145, lr_0 = 8.9978e-04
Validation prc-auc = 0.210441
Epoch 70
Validation prc-auc = 0.203410
Epoch 71
Validation prc-auc = 0.230668
Epoch 72
Validation prc-auc = 0.244779
Epoch 73
Validation prc-auc = 0.196868
Epoch 74
Loss = 1.0429e-01, PNorm = 83.8143, GNorm = 3.5757, lr_0 = 8.9209e-04
Validation prc-auc = 0.252263
Epoch 75
Validation prc-auc = 0.282624
Epoch 76
Validation prc-auc = 0.206495
Epoch 77
Validation prc-auc = 0.203314
Epoch 78
Validation prc-auc = 0.246712
Epoch 79
Loss = 2.2613e-02, PNorm = 84.1355, GNorm = 1.1337, lr_0 = 8.8447e-04
Validation prc-auc = 0.296338
Epoch 80
Validation prc-auc = 0.259639
Epoch 81
Validation prc-auc = 0.227287
Epoch 82
Validation prc-auc = 0.263878
Epoch 83
Validation prc-auc = 0.265886
Epoch 84
Loss = 1.8350e-02, PNorm = 84.4276, GNorm = 0.6847, lr_0 = 8.7691e-04
Validation prc-auc = 0.221338
Epoch 85
Validation prc-auc = 0.228542
Epoch 86
Validation prc-auc = 0.236924
Epoch 87
Validation prc-auc = 0.244105
Epoch 88
Validation prc-auc = 0.234104
Epoch 89
Loss = 4.6554e-02, PNorm = 84.6735, GNorm = 0.5354, lr_0 = 8.6941e-04
Validation prc-auc = 0.192770
Epoch 90
Validation prc-auc = 0.203480
Epoch 91
Validation prc-auc = 0.241460
Epoch 92
Validation prc-auc = 0.254533
Epoch 93
Validation prc-auc = 0.243593
Epoch 94
Loss = 3.3112e-02, PNorm = 84.9733, GNorm = 0.6421, lr_0 = 8.6198e-04
Validation prc-auc = 0.230764
Epoch 95
Validation prc-auc = 0.230331
Epoch 96
Validation prc-auc = 0.205606
Epoch 97
Validation prc-auc = 0.178229
Epoch 98
Validation prc-auc = 0.182735
Epoch 99
Loss = 1.0864e-02, PNorm = 85.2441, GNorm = 0.1917, lr_0 = 8.5461e-04
Validation prc-auc = 0.213917
Epoch 100
Validation prc-auc = 0.239236
Epoch 101
Validation prc-auc = 0.281919
Epoch 102
Validation prc-auc = 0.287679
Epoch 103
Validation prc-auc = 0.255842
Epoch 104
Loss = 9.8627e-03, PNorm = 85.4615, GNorm = 0.2517, lr_0 = 8.4730e-04
Validation prc-auc = 0.231141
Epoch 105
Validation prc-auc = 0.227071
Epoch 106
Validation prc-auc = 0.257809
Epoch 107
Validation prc-auc = 0.267455
Epoch 108
Validation prc-auc = 0.277898
Epoch 109
Loss = 4.2696e-03, PNorm = 85.6242, GNorm = 0.1194, lr_0 = 8.4006e-04
Validation prc-auc = 0.281482
Epoch 110
Validation prc-auc = 0.328216
Epoch 111
Validation prc-auc = 0.310008
Epoch 112
Validation prc-auc = 0.267946
Epoch 113
Validation prc-auc = 0.247763
Epoch 114
Loss = 1.0243e-02, PNorm = 85.7935, GNorm = 0.4677, lr_0 = 8.3288e-04
Validation prc-auc = 0.261461
Epoch 115
Validation prc-auc = 0.242210
Epoch 116
Validation prc-auc = 0.242133
Epoch 117
Validation prc-auc = 0.276419
Epoch 118
Validation prc-auc = 0.200080
Epoch 119
Loss = 7.4002e-02, PNorm = 86.1586, GNorm = 2.8840, lr_0 = 8.2576e-04
Validation prc-auc = 0.180384
Epoch 120
Validation prc-auc = 0.285562
Epoch 121
Validation prc-auc = 0.311209
Epoch 122
Validation prc-auc = 0.294940
Epoch 123
Validation prc-auc = 0.249885
Epoch 124
Loss = 1.7930e-01, PNorm = 86.5154, GNorm = 0.8035, lr_0 = 8.1870e-04
Validation prc-auc = 0.255555
Epoch 125
Validation prc-auc = 0.230887
Epoch 126
Validation prc-auc = 0.228909
Epoch 127
Validation prc-auc = 0.264813
Epoch 128
Validation prc-auc = 0.275429
Epoch 129
Loss = 1.8357e-01, PNorm = 86.8617, GNorm = 4.5801, lr_0 = 8.1170e-04
Validation prc-auc = 0.229094
Epoch 130
Validation prc-auc = 0.194775
Epoch 131
Validation prc-auc = 0.168031
Epoch 132
Validation prc-auc = 0.191172
Epoch 133
Validation prc-auc = 0.227478
Epoch 134
Loss = 1.1342e-02, PNorm = 87.2355, GNorm = 0.2556, lr_0 = 8.0476e-04
Validation prc-auc = 0.246038
Epoch 135
Validation prc-auc = 0.255616
Epoch 136
Validation prc-auc = 0.254731
Epoch 137
Validation prc-auc = 0.249726
Epoch 138
Validation prc-auc = 0.245840
Epoch 139
Loss = 6.2263e-03, PNorm = 87.4927, GNorm = 0.0195, lr_0 = 7.9788e-04
Validation prc-auc = 0.244570
Epoch 140
Validation prc-auc = 0.235615
Epoch 141
Validation prc-auc = 0.220272
Epoch 142
Validation prc-auc = 0.208459
Epoch 143
Validation prc-auc = 0.200123
Epoch 144
Loss = 4.0279e-03, PNorm = 87.6342, GNorm = 0.1075, lr_0 = 7.9106e-04
Validation prc-auc = 0.192568
Epoch 145
Validation prc-auc = 0.186821
Epoch 146
Validation prc-auc = 0.184707
Epoch 147
Validation prc-auc = 0.195172
Epoch 148
Validation prc-auc = 0.213246
Epoch 149
Loss = 3.8667e-03, PNorm = 87.7297, GNorm = 0.1326, lr_0 = 7.8430e-04
Validation prc-auc = 0.215438
Epoch 150
Validation prc-auc = 0.202191
Epoch 151
Validation prc-auc = 0.202999
Epoch 152
Validation prc-auc = 0.195512
Epoch 153
Validation prc-auc = 0.212758
Epoch 154
Loss = 3.2520e-03, PNorm = 87.8064, GNorm = 0.0373, lr_0 = 7.7759e-04
Validation prc-auc = 0.237215
Epoch 155
Validation prc-auc = 0.244448
Epoch 156
Validation prc-auc = 0.251906
Epoch 157
Validation prc-auc = 0.237463
Epoch 158
Validation prc-auc = 0.221265
Epoch 159
Loss = 1.5139e-03, PNorm = 87.8657, GNorm = 0.0233, lr_0 = 7.7095e-04
Validation prc-auc = 0.219616
Epoch 160
Validation prc-auc = 0.210079
Epoch 161
Validation prc-auc = 0.208258
Epoch 162
Validation prc-auc = 0.207564
Epoch 163
Validation prc-auc = 0.211004
Epoch 164
Loss = 2.6689e-03, PNorm = 87.9153, GNorm = 0.0170, lr_0 = 7.6436e-04
Validation prc-auc = 0.215004
Epoch 165
Validation prc-auc = 0.227429
Epoch 166
Validation prc-auc = 0.244716
Epoch 167
Validation prc-auc = 0.243009
Epoch 168
Validation prc-auc = 0.246214
Epoch 169
Loss = 1.8959e-03, PNorm = 87.9682, GNorm = 0.0144, lr_0 = 7.5782e-04
Validation prc-auc = 0.244058
Epoch 170
Validation prc-auc = 0.246435
Epoch 171
Validation prc-auc = 0.237189
Epoch 172
Validation prc-auc = 0.231434
Epoch 173
Validation prc-auc = 0.230957
Epoch 174
Loss = 3.2964e-03, PNorm = 88.0205, GNorm = 0.2178, lr_0 = 7.5134e-04
Validation prc-auc = 0.230271
Epoch 175
Validation prc-auc = 0.236124
Epoch 176
Validation prc-auc = 0.234886
Epoch 177
Validation prc-auc = 0.233873
Epoch 178
Validation prc-auc = 0.238762
Epoch 179
Loss = 9.4037e-04, PNorm = 88.0521, GNorm = 0.0239, lr_0 = 7.4492e-04
Validation prc-auc = 0.240527
Epoch 180
Validation prc-auc = 0.241138
Epoch 181
Validation prc-auc = 0.238063
Epoch 182
Validation prc-auc = 0.238238
Epoch 183
Validation prc-auc = 0.234292
Epoch 184
Loss = 4.1477e-04, PNorm = 88.0769, GNorm = 0.0094, lr_0 = 7.3855e-04
Validation prc-auc = 0.234197
Epoch 185
Validation prc-auc = 0.234250
Epoch 186
Validation prc-auc = 0.233476
Epoch 187
Validation prc-auc = 0.233148
Epoch 188
Validation prc-auc = 0.233055
Epoch 189
Loss = 6.4001e-04, PNorm = 88.0948, GNorm = 0.0178, lr_0 = 7.3224e-04
Validation prc-auc = 0.232867
Epoch 190
Validation prc-auc = 0.233167
Epoch 191
Validation prc-auc = 0.233055
Epoch 192
Validation prc-auc = 0.236434
Epoch 193
Validation prc-auc = 0.236434
Epoch 194
Loss = 3.1285e-04, PNorm = 88.1079, GNorm = 0.0107, lr_0 = 7.2598e-04
Validation prc-auc = 0.236260
Epoch 195
Validation prc-auc = 0.235655
Epoch 196
Validation prc-auc = 0.236170
Epoch 197
Validation prc-auc = 0.239206
Epoch 198
Validation prc-auc = 0.239687
Epoch 199
Loss = 1.6953e-04, PNorm = 88.1196, GNorm = 0.0048, lr_0 = 7.1977e-04
Validation prc-auc = 0.244501
Epoch 200
Validation prc-auc = 0.244501
Epoch 201
Validation prc-auc = 0.244448
Epoch 202
Validation prc-auc = 0.244448
Epoch 203
Validation prc-auc = 0.244448
Epoch 204
Loss = 1.7062e-04, PNorm = 88.1297, GNorm = 0.0036, lr_0 = 7.1362e-04
Validation prc-auc = 0.248064
Epoch 205
Validation prc-auc = 0.248064
Epoch 206
Validation prc-auc = 0.248155
Epoch 207
Validation prc-auc = 0.243041
Epoch 208
Validation prc-auc = 0.243115
Epoch 209
Loss = 3.0831e-04, PNorm = 88.1389, GNorm = 0.0104, lr_0 = 7.0752e-04
Validation prc-auc = 0.243065
Epoch 210
Validation prc-auc = 0.243065
Epoch 211
Validation prc-auc = 0.242946
Epoch 212
Validation prc-auc = 0.242946
Epoch 213
Validation prc-auc = 0.243838
Epoch 214
Loss = 3.3854e-04, PNorm = 88.1500, GNorm = 0.0195, lr_0 = 7.0147e-04
Validation prc-auc = 0.243941
Epoch 215
Validation prc-auc = 0.243829
Epoch 216
Validation prc-auc = 0.243689
Epoch 217
Validation prc-auc = 0.243780
Epoch 218
Validation prc-auc = 0.243670
Epoch 219
Loss = 1.2852e-04, PNorm = 88.1621, GNorm = 0.0031, lr_0 = 6.9548e-04
Validation prc-auc = 0.243670
Epoch 220
Validation prc-auc = 0.243670
Epoch 221
Validation prc-auc = 0.243344
Epoch 222
Validation prc-auc = 0.243896
Epoch 223
Validation prc-auc = 0.247709
Epoch 224
Loss = 9.1210e-05, PNorm = 88.1722, GNorm = 0.0038, lr_0 = 6.8953e-04
Validation prc-auc = 0.247617
Epoch 225
Validation prc-auc = 0.247228
Epoch 226
Validation prc-auc = 0.247228
Epoch 227
Validation prc-auc = 0.247391
Epoch 228
Validation prc-auc = 0.247207
Epoch 229
Loss = 1.6087e-04, PNorm = 88.1795, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.247207
Epoch 230
Validation prc-auc = 0.243668
Epoch 231
Validation prc-auc = 0.243751
Epoch 232
Validation prc-auc = 0.243055
Epoch 233
Validation prc-auc = 0.243023
Epoch 234
Loss = 3.0499e-04, PNorm = 88.1857, GNorm = 0.0040, lr_0 = 6.7779e-04
Validation prc-auc = 0.242620
Epoch 235
Validation prc-auc = 0.242620
Epoch 236
Validation prc-auc = 0.246080
Epoch 237
Validation prc-auc = 0.245971
Epoch 238
Validation prc-auc = 0.246027
Epoch 239
Loss = 1.0634e-04, PNorm = 88.1956, GNorm = 0.0032, lr_0 = 6.7200e-04
Validation prc-auc = 0.246119
Epoch 240
Validation prc-auc = 0.246074
Epoch 241
Validation prc-auc = 0.246182
Epoch 242
Validation prc-auc = 0.246800
Epoch 243
Validation prc-auc = 0.246733
Epoch 244
Loss = 7.6656e-05, PNorm = 88.2041, GNorm = 0.0031, lr_0 = 6.6625e-04
Validation prc-auc = 0.246733
Epoch 245
Validation prc-auc = 0.250039
Epoch 246
Validation prc-auc = 0.250039
Epoch 247
Validation prc-auc = 0.250039
Epoch 248
Validation prc-auc = 0.252690
Epoch 249
Loss = 8.8182e-05, PNorm = 88.2116, GNorm = 0.0031, lr_0 = 6.6056e-04
Validation prc-auc = 0.252536
Epoch 250
Validation prc-auc = 0.252603
Epoch 251
Validation prc-auc = 0.252820
Epoch 252
Validation prc-auc = 0.252596
Epoch 253
Validation prc-auc = 0.252877
Epoch 254
Loss = 1.1104e-04, PNorm = 88.2205, GNorm = 0.0021, lr_0 = 6.5491e-04
Validation prc-auc = 0.252811
Epoch 255
Validation prc-auc = 0.252208
Epoch 256
Validation prc-auc = 0.252420
Epoch 257
Validation prc-auc = 0.251804
Epoch 258
Validation prc-auc = 0.251741
Epoch 259
Loss = 8.3043e-05, PNorm = 88.2293, GNorm = 0.0040, lr_0 = 6.4931e-04
Validation prc-auc = 0.251280
Epoch 260
Validation prc-auc = 0.250420
Epoch 261
Validation prc-auc = 0.250420
Epoch 262
Validation prc-auc = 0.250327
Epoch 263
Validation prc-auc = 0.250420
Epoch 264
Loss = 4.5222e-05, PNorm = 88.2363, GNorm = 0.0010, lr_0 = 6.4376e-04
Validation prc-auc = 0.250901
Epoch 265
Validation prc-auc = 0.251021
Epoch 266
Validation prc-auc = 0.250712
Epoch 267
Validation prc-auc = 0.250231
Epoch 268
Validation prc-auc = 0.250298
Epoch 269
Loss = 4.5276e-05, PNorm = 88.2420, GNorm = 0.0015, lr_0 = 6.3826e-04
Validation prc-auc = 0.254069
Epoch 270
Validation prc-auc = 0.253805
Epoch 271
Validation prc-auc = 0.253968
Epoch 272
Validation prc-auc = 0.253546
Epoch 273
Validation prc-auc = 0.253968
Epoch 274
Loss = 5.4851e-05, PNorm = 88.2473, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.254352
Epoch 275
Validation prc-auc = 0.254282
Epoch 276
Validation prc-auc = 0.254763
Epoch 277
Validation prc-auc = 0.254830
Epoch 278
Validation prc-auc = 0.254807
Epoch 279
Loss = 3.4437e-05, PNorm = 88.2520, GNorm = 0.0017, lr_0 = 6.2739e-04
Validation prc-auc = 0.255005
Epoch 280
Validation prc-auc = 0.255532
Epoch 281
Validation prc-auc = 0.255532
Epoch 282
Validation prc-auc = 0.255348
Epoch 283
Validation prc-auc = 0.255501
Epoch 284
Loss = 6.7237e-05, PNorm = 88.2572, GNorm = 0.0009, lr_0 = 6.2203e-04
Validation prc-auc = 0.255957
Epoch 285
Validation prc-auc = 0.256053
Epoch 286
Validation prc-auc = 0.260197
Epoch 287
Validation prc-auc = 0.260197
Epoch 288
Validation prc-auc = 0.266863
Epoch 289
Loss = 2.3074e-05, PNorm = 88.2622, GNorm = 0.0009, lr_0 = 6.1671e-04
Validation prc-auc = 0.266264
Epoch 290
Validation prc-auc = 0.266060
Epoch 291
Validation prc-auc = 0.266046
Epoch 292
Validation prc-auc = 0.265635
Epoch 293
Validation prc-auc = 0.265446
Epoch 294
Loss = 3.0786e-05, PNorm = 88.2662, GNorm = 0.0002, lr_0 = 6.1144e-04
Validation prc-auc = 0.265517
Epoch 295
Validation prc-auc = 0.265998
Epoch 296
Validation prc-auc = 0.266091
Epoch 297
Validation prc-auc = 0.266091
Epoch 298
Validation prc-auc = 0.266174
Epoch 299
Loss = 3.7240e-05, PNorm = 88.2701, GNorm = 0.0037, lr_0 = 6.0621e-04
Validation prc-auc = 0.266137
Model 0 best validation prc-auc = 0.487608 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.528662
Ensemble test prc-auc = 0.528662
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.420661
Epoch 1
Validation prc-auc = 0.457936
Epoch 2
Validation prc-auc = 0.458228
Epoch 3
Validation prc-auc = 0.461219
Epoch 4
Loss = 6.7879e-01, PNorm = 78.5484, GNorm = 9.4250, lr_0 = 6.5000e-04
Validation prc-auc = 0.457434
Epoch 5
Validation prc-auc = 0.485822
Epoch 6
Validation prc-auc = 0.469495
Epoch 7
Validation prc-auc = 0.474223
Epoch 8
Validation prc-auc = 0.490912
Epoch 9
Loss = 6.7939e-01, PNorm = 78.7767, GNorm = 12.4080, lr_0 = 9.9743e-04
Validation prc-auc = 0.337378
Epoch 10
Validation prc-auc = 0.301258
Epoch 11
Validation prc-auc = 0.259962
Epoch 12
Validation prc-auc = 0.370990
Epoch 13
Validation prc-auc = 0.403101
Epoch 14
Loss = 6.1304e-01, PNorm = 79.0637, GNorm = 0.8893, lr_0 = 9.8890e-04
Validation prc-auc = 0.309827
Epoch 15
Validation prc-auc = 0.291735
Epoch 16
Validation prc-auc = 0.307380
Epoch 17
Validation prc-auc = 0.308140
Epoch 18
Validation prc-auc = 0.428703
Epoch 19
Loss = 5.2677e-01, PNorm = 79.3599, GNorm = 2.3717, lr_0 = 9.8045e-04
Validation prc-auc = 0.404927
Epoch 20
Validation prc-auc = 0.361680
Epoch 21
Validation prc-auc = 0.450644
Epoch 22
Validation prc-auc = 0.426842
Epoch 23
Validation prc-auc = 0.392685
Epoch 24
Loss = 4.6043e-01, PNorm = 79.6595, GNorm = 4.8828, lr_0 = 9.7207e-04
Validation prc-auc = 0.349631
Epoch 25
Validation prc-auc = 0.326684
Epoch 26
Validation prc-auc = 0.319028
Epoch 27
Validation prc-auc = 0.348372
Epoch 28
Validation prc-auc = 0.288309
Epoch 29
Loss = 3.7091e-01, PNorm = 79.9930, GNorm = 1.4221, lr_0 = 9.6376e-04
Validation prc-auc = 0.297019
Epoch 30
Validation prc-auc = 0.285389
Epoch 31
Validation prc-auc = 0.314174
Epoch 32
Validation prc-auc = 0.328227
Epoch 33
Validation prc-auc = 0.425654
Epoch 34
Loss = 3.2539e-01, PNorm = 80.3645, GNorm = 2.5136, lr_0 = 9.5552e-04
Validation prc-auc = 0.451703
Epoch 35
Validation prc-auc = 0.439275
Epoch 36
Validation prc-auc = 0.344190
Epoch 37
Validation prc-auc = 0.233892
Epoch 38
Validation prc-auc = 0.332029
Epoch 39
Loss = 2.5436e-01, PNorm = 80.7576, GNorm = 4.8816, lr_0 = 9.4735e-04
Validation prc-auc = 0.360506
Epoch 40
Validation prc-auc = 0.365633
Epoch 41
Validation prc-auc = 0.363018
Epoch 42
Validation prc-auc = 0.362571
Epoch 43
Validation prc-auc = 0.280235
Epoch 44
Loss = 2.9132e-01, PNorm = 81.2063, GNorm = 4.9521, lr_0 = 9.3925e-04
Validation prc-auc = 0.313984
Epoch 45
Validation prc-auc = 0.341950
Epoch 46
Validation prc-auc = 0.338457
Epoch 47
Validation prc-auc = 0.309914
Epoch 48
Validation prc-auc = 0.190773
Epoch 49
Loss = 2.0441e-01, PNorm = 81.7002, GNorm = 1.1314, lr_0 = 9.3122e-04
Validation prc-auc = 0.162704
Epoch 50
Validation prc-auc = 0.248556
Epoch 51
Validation prc-auc = 0.297140
Epoch 52
Validation prc-auc = 0.301968
Epoch 53
Validation prc-auc = 0.312886
Epoch 54
Loss = 1.0181e-01, PNorm = 82.1900, GNorm = 0.4953, lr_0 = 9.2326e-04
Validation prc-auc = 0.186690
Epoch 55
Validation prc-auc = 0.328122
Epoch 56
Validation prc-auc = 0.225372
Epoch 57
Validation prc-auc = 0.280809
Epoch 58
Validation prc-auc = 0.329574
Epoch 59
Loss = 3.3973e-01, PNorm = 82.6580, GNorm = 4.7337, lr_0 = 9.1537e-04
Validation prc-auc = 0.346119
Epoch 60
Validation prc-auc = 0.279217
Epoch 61
Validation prc-auc = 0.288589
Epoch 62
Validation prc-auc = 0.334008
Epoch 63
Validation prc-auc = 0.298605
Epoch 64
Loss = 8.9413e-02, PNorm = 83.0737, GNorm = 0.3650, lr_0 = 9.0754e-04
Validation prc-auc = 0.307533
Epoch 65
Validation prc-auc = 0.287666
Epoch 66
Validation prc-auc = 0.270246
Epoch 67
Validation prc-auc = 0.226745
Epoch 68
Validation prc-auc = 0.221687
Epoch 69
Loss = 7.1962e-02, PNorm = 83.4600, GNorm = 1.0944, lr_0 = 8.9978e-04
Validation prc-auc = 0.226417
Epoch 70
Validation prc-auc = 0.256125
Epoch 71
Validation prc-auc = 0.272778
Epoch 72
Validation prc-auc = 0.247839
Epoch 73
Validation prc-auc = 0.248686
Epoch 74
Loss = 6.4427e-02, PNorm = 83.7716, GNorm = 0.3512, lr_0 = 8.9209e-04
Validation prc-auc = 0.311930
Epoch 75
Validation prc-auc = 0.320713
Epoch 76
Validation prc-auc = 0.306418
Epoch 77
Validation prc-auc = 0.293195
Epoch 78
Validation prc-auc = 0.283546
Epoch 79
Loss = 5.7602e-02, PNorm = 84.0756, GNorm = 0.7084, lr_0 = 8.8447e-04
Validation prc-auc = 0.250902
Epoch 80
Validation prc-auc = 0.233777
Epoch 81
Validation prc-auc = 0.224770
Epoch 82
Validation prc-auc = 0.240630
Epoch 83
Validation prc-auc = 0.257950
Epoch 84
Loss = 2.8243e-02, PNorm = 84.3772, GNorm = 0.0990, lr_0 = 8.7691e-04
Validation prc-auc = 0.239297
Epoch 85
Validation prc-auc = 0.275845
Epoch 86
Validation prc-auc = 0.296512
Epoch 87
Validation prc-auc = 0.275751
Epoch 88
Validation prc-auc = 0.281087
Epoch 89
Loss = 1.3489e-02, PNorm = 84.6147, GNorm = 0.8700, lr_0 = 8.6941e-04
Validation prc-auc = 0.300239
Epoch 90
Validation prc-auc = 0.307408
Epoch 91
Validation prc-auc = 0.305317
Epoch 92
Validation prc-auc = 0.283052
Epoch 93
Validation prc-auc = 0.263338
Epoch 94
Loss = 4.1247e-02, PNorm = 84.8298, GNorm = 0.7225, lr_0 = 8.6198e-04
Validation prc-auc = 0.242708
Epoch 95
Validation prc-auc = 0.217276
Epoch 96
Validation prc-auc = 0.243376
Epoch 97
Validation prc-auc = 0.294436
Epoch 98
Validation prc-auc = 0.307211
Epoch 99
Loss = 1.7993e-02, PNorm = 85.0559, GNorm = 0.5066, lr_0 = 8.5461e-04
Validation prc-auc = 0.292837
Epoch 100
Validation prc-auc = 0.282350
Epoch 101
Validation prc-auc = 0.246013
Epoch 102
Validation prc-auc = 0.248986
Epoch 103
Validation prc-auc = 0.263404
Epoch 104
Loss = 3.3377e-02, PNorm = 85.2747, GNorm = 0.1569, lr_0 = 8.4730e-04
Validation prc-auc = 0.336663
Epoch 105
Validation prc-auc = 0.346621
Epoch 106
Validation prc-auc = 0.342681
Epoch 107
Validation prc-auc = 0.334539
Epoch 108
Validation prc-auc = 0.328910
Epoch 109
Loss = 2.5633e-02, PNorm = 85.4632, GNorm = 1.8693, lr_0 = 8.4006e-04
Validation prc-auc = 0.302107
Epoch 110
Validation prc-auc = 0.211599
Epoch 111
Validation prc-auc = 0.264865
Epoch 112
Validation prc-auc = 0.275362
Epoch 113
Validation prc-auc = 0.229602
Epoch 114
Loss = 6.8220e-02, PNorm = 85.6689, GNorm = 2.2773, lr_0 = 8.3288e-04
Validation prc-auc = 0.275315
Epoch 115
Validation prc-auc = 0.304039
Epoch 116
Validation prc-auc = 0.255938
Epoch 117
Validation prc-auc = 0.233755
Epoch 118
Validation prc-auc = 0.216920
Epoch 119
Loss = 6.1016e-02, PNorm = 85.9954, GNorm = 0.2323, lr_0 = 8.2576e-04
Validation prc-auc = 0.234682
Epoch 120
Validation prc-auc = 0.222558
Epoch 121
Validation prc-auc = 0.219669
Epoch 122
Validation prc-auc = 0.237908
Epoch 123
Validation prc-auc = 0.228341
Epoch 124
Loss = 2.3777e-02, PNorm = 86.2930, GNorm = 0.1455, lr_0 = 8.1870e-04
Validation prc-auc = 0.210794
Epoch 125
Validation prc-auc = 0.199932
Epoch 126
Validation prc-auc = 0.221785
Epoch 127
Validation prc-auc = 0.228188
Epoch 128
Validation prc-auc = 0.229822
Epoch 129
Loss = 2.1434e-02, PNorm = 86.5291, GNorm = 0.5616, lr_0 = 8.1170e-04
Validation prc-auc = 0.243109
Epoch 130
Validation prc-auc = 0.236189
Epoch 131
Validation prc-auc = 0.207302
Epoch 132
Validation prc-auc = 0.193973
Epoch 133
Validation prc-auc = 0.217384
Epoch 134
Loss = 1.2687e-01, PNorm = 86.7778, GNorm = 2.5584, lr_0 = 8.0476e-04
Validation prc-auc = 0.221427
Epoch 135
Validation prc-auc = 0.225822
Epoch 136
Validation prc-auc = 0.221941
Epoch 137
Validation prc-auc = 0.224318
Epoch 138
Validation prc-auc = 0.235118
Epoch 139
Loss = 1.2207e-02, PNorm = 86.9729, GNorm = 0.0883, lr_0 = 7.9788e-04
Validation prc-auc = 0.241458
Epoch 140
Validation prc-auc = 0.247542
Epoch 141
Validation prc-auc = 0.255155
Epoch 142
Validation prc-auc = 0.255968
Epoch 143
Validation prc-auc = 0.274385
Epoch 144
Loss = 3.4209e-02, PNorm = 87.1143, GNorm = 0.5921, lr_0 = 7.9106e-04
Validation prc-auc = 0.285068
Epoch 145
Validation prc-auc = 0.279809
Epoch 146
Validation prc-auc = 0.269946
Epoch 147
Validation prc-auc = 0.262443
Epoch 148
Validation prc-auc = 0.239521
Epoch 149
Loss = 7.4511e-03, PNorm = 87.2699, GNorm = 0.1724, lr_0 = 7.8430e-04
Validation prc-auc = 0.227633
Epoch 150
Validation prc-auc = 0.218389
Epoch 151
Validation prc-auc = 0.214915
Epoch 152
Validation prc-auc = 0.260338
Epoch 153
Validation prc-auc = 0.270727
Epoch 154
Loss = 4.3829e-03, PNorm = 87.4757, GNorm = 0.0986, lr_0 = 7.7759e-04
Validation prc-auc = 0.248127
Epoch 155
Validation prc-auc = 0.271496
Epoch 156
Validation prc-auc = 0.279499
Epoch 157
Validation prc-auc = 0.273700
Epoch 158
Validation prc-auc = 0.233976
Epoch 159
Loss = 3.6222e-02, PNorm = 87.6901, GNorm = 0.5669, lr_0 = 7.7095e-04
Validation prc-auc = 0.181578
Epoch 160
Validation prc-auc = 0.197170
Epoch 161
Validation prc-auc = 0.211570
Epoch 162
Validation prc-auc = 0.206212
Epoch 163
Validation prc-auc = 0.215330
Epoch 164
Loss = 1.2465e-01, PNorm = 87.9610, GNorm = 2.2373, lr_0 = 7.6436e-04
Validation prc-auc = 0.265177
Epoch 165
Validation prc-auc = 0.274934
Epoch 166
Validation prc-auc = 0.231637
Epoch 167
Validation prc-auc = 0.204012
Epoch 168
Validation prc-auc = 0.189641
Epoch 169
Loss = 3.4128e-02, PNorm = 88.2434, GNorm = 0.3778, lr_0 = 7.5782e-04
Validation prc-auc = 0.208085
Epoch 170
Validation prc-auc = 0.257314
Epoch 171
Validation prc-auc = 0.319175
Epoch 172
Validation prc-auc = 0.315239
Epoch 173
Validation prc-auc = 0.280051
Epoch 174
Loss = 6.4917e-03, PNorm = 88.5055, GNorm = 0.1110, lr_0 = 7.5134e-04
Validation prc-auc = 0.211732
Epoch 175
Validation prc-auc = 0.184104
Epoch 176
Validation prc-auc = 0.207875
Epoch 177
Validation prc-auc = 0.237450
Epoch 178
Validation prc-auc = 0.266153
Epoch 179
Loss = 6.7845e-03, PNorm = 88.6905, GNorm = 0.0565, lr_0 = 7.4492e-04
Validation prc-auc = 0.292587
Epoch 180
Validation prc-auc = 0.296856
Epoch 181
Validation prc-auc = 0.306850
Epoch 182
Validation prc-auc = 0.286291
Epoch 183
Validation prc-auc = 0.282767
Epoch 184
Loss = 3.5275e-03, PNorm = 88.8265, GNorm = 0.0604, lr_0 = 7.3855e-04
Validation prc-auc = 0.268273
Epoch 185
Validation prc-auc = 0.266838
Epoch 186
Validation prc-auc = 0.263637
Epoch 187
Validation prc-auc = 0.261352
Epoch 188
Validation prc-auc = 0.267484
Epoch 189
Loss = 2.5836e-03, PNorm = 88.9277, GNorm = 0.0224, lr_0 = 7.3224e-04
Validation prc-auc = 0.267054
Epoch 190
Validation prc-auc = 0.268399
Epoch 191
Validation prc-auc = 0.276735
Epoch 192
Validation prc-auc = 0.287725
Epoch 193
Validation prc-auc = 0.288679
Epoch 194
Loss = 1.9950e-03, PNorm = 88.9891, GNorm = 0.0316, lr_0 = 7.2598e-04
Validation prc-auc = 0.289114
Epoch 195
Validation prc-auc = 0.288929
Epoch 196
Validation prc-auc = 0.282456
Epoch 197
Validation prc-auc = 0.272728
Epoch 198
Validation prc-auc = 0.271680
Epoch 199
Loss = 8.0198e-04, PNorm = 89.0349, GNorm = 0.0207, lr_0 = 7.1977e-04
Validation prc-auc = 0.271598
Epoch 200
Validation prc-auc = 0.271760
Epoch 201
Validation prc-auc = 0.271311
Epoch 202
Validation prc-auc = 0.271315
Epoch 203
Validation prc-auc = 0.271311
Epoch 204
Loss = 1.0285e-03, PNorm = 89.0619, GNorm = 0.1913, lr_0 = 7.1362e-04
Validation prc-auc = 0.271392
Epoch 205
Validation prc-auc = 0.278023
Epoch 206
Validation prc-auc = 0.281764
Epoch 207
Validation prc-auc = 0.281656
Epoch 208
Validation prc-auc = 0.283690
Epoch 209
Loss = 1.4612e-03, PNorm = 89.0858, GNorm = 0.0122, lr_0 = 7.0752e-04
Validation prc-auc = 0.283638
Epoch 210
Validation prc-auc = 0.283437
Epoch 211
Validation prc-auc = 0.281394
Epoch 212
Validation prc-auc = 0.279675
Epoch 213
Validation prc-auc = 0.279458
Epoch 214
Loss = 4.5339e-04, PNorm = 89.1154, GNorm = 0.0102, lr_0 = 7.0147e-04
Validation prc-auc = 0.279104
Epoch 215
Validation prc-auc = 0.279294
Epoch 216
Validation prc-auc = 0.277534
Epoch 217
Validation prc-auc = 0.277420
Epoch 218
Validation prc-auc = 0.277356
Epoch 219
Loss = 1.2988e-03, PNorm = 89.1366, GNorm = 0.0905, lr_0 = 6.9548e-04
Validation prc-auc = 0.277290
Epoch 220
Validation prc-auc = 0.277382
Epoch 221
Validation prc-auc = 0.277386
Epoch 222
Validation prc-auc = 0.277469
Epoch 223
Validation prc-auc = 0.277310
Epoch 224
Loss = 2.4854e-04, PNorm = 89.1491, GNorm = 0.0055, lr_0 = 6.8953e-04
Validation prc-auc = 0.277302
Epoch 225
Validation prc-auc = 0.277302
Epoch 226
Validation prc-auc = 0.277252
Epoch 227
Validation prc-auc = 0.277247
Epoch 228
Validation prc-auc = 0.277420
Epoch 229
Loss = 1.6646e-04, PNorm = 89.1637, GNorm = 0.0050, lr_0 = 6.8364e-04
Validation prc-auc = 0.277234
Epoch 230
Validation prc-auc = 0.277172
Epoch 231
Validation prc-auc = 0.277155
Epoch 232
Validation prc-auc = 0.277040
Epoch 233
Validation prc-auc = 0.283128
Epoch 234
Loss = 1.7735e-04, PNorm = 89.1747, GNorm = 0.0067, lr_0 = 6.7779e-04
Validation prc-auc = 0.283067
Epoch 235
Validation prc-auc = 0.283079
Epoch 236
Validation prc-auc = 0.283030
Epoch 237
Validation prc-auc = 0.281935
Epoch 238
Validation prc-auc = 0.281935
Epoch 239
Loss = 1.2403e-04, PNorm = 89.1835, GNorm = 0.0077, lr_0 = 6.7200e-04
Validation prc-auc = 0.281935
Epoch 240
Validation prc-auc = 0.281935
Epoch 241
Validation prc-auc = 0.281935
Epoch 242
Validation prc-auc = 0.282439
Epoch 243
Validation prc-auc = 0.282439
Epoch 244
Loss = 1.6262e-04, PNorm = 89.1912, GNorm = 0.0020, lr_0 = 6.6625e-04
Validation prc-auc = 0.282439
Epoch 245
Validation prc-auc = 0.282390
Epoch 246
Validation prc-auc = 0.282390
Epoch 247
Validation prc-auc = 0.282342
Epoch 248
Validation prc-auc = 0.282403
Epoch 249
Loss = 1.6689e-04, PNorm = 89.1992, GNorm = 0.0021, lr_0 = 6.6056e-04
Validation prc-auc = 0.282403
Epoch 250
Validation prc-auc = 0.282470
Epoch 251
Validation prc-auc = 0.282424
Epoch 252
Validation prc-auc = 0.282487
Epoch 253
Validation prc-auc = 0.282541
Epoch 254
Loss = 1.0802e-04, PNorm = 89.2060, GNorm = 0.0025, lr_0 = 6.5491e-04
Validation prc-auc = 0.282478
Epoch 255
Validation prc-auc = 0.282478
Epoch 256
Validation prc-auc = 0.282416
Epoch 257
Validation prc-auc = 0.282416
Epoch 258
Validation prc-auc = 0.282355
Epoch 259
Loss = 8.0309e-05, PNorm = 89.2132, GNorm = 0.0017, lr_0 = 6.4931e-04
Validation prc-auc = 0.282355
Epoch 260
Validation prc-auc = 0.282355
Epoch 261
Validation prc-auc = 0.282948
Epoch 262
Validation prc-auc = 0.282948
Epoch 263
Validation prc-auc = 0.282948
Epoch 264
Loss = 1.2134e-04, PNorm = 89.2205, GNorm = 0.0028, lr_0 = 6.4376e-04
Validation prc-auc = 0.283067
Epoch 265
Validation prc-auc = 0.283067
Epoch 266
Validation prc-auc = 0.283067
Epoch 267
Validation prc-auc = 0.283067
Epoch 268
Validation prc-auc = 0.283067
Epoch 269
Loss = 1.1993e-04, PNorm = 89.2285, GNorm = 0.0048, lr_0 = 6.3826e-04
Validation prc-auc = 0.283122
Epoch 270
Validation prc-auc = 0.283122
Epoch 271
Validation prc-auc = 0.283183
Epoch 272
Validation prc-auc = 0.283183
Epoch 273
Validation prc-auc = 0.283183
Epoch 274
Loss = 5.0126e-05, PNorm = 89.2354, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.282589
Epoch 275
Validation prc-auc = 0.282635
Epoch 276
Validation prc-auc = 0.282635
Epoch 277
Validation prc-auc = 0.282635
Epoch 278
Validation prc-auc = 0.282589
Epoch 279
Loss = 5.5134e-05, PNorm = 89.2421, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation prc-auc = 0.282711
Epoch 280
Validation prc-auc = 0.282711
Epoch 281
Validation prc-auc = 0.282711
Epoch 282
Validation prc-auc = 0.282711
Epoch 283
Validation prc-auc = 0.282760
Epoch 284
Loss = 5.5362e-05, PNorm = 89.2500, GNorm = 0.0021, lr_0 = 6.2203e-04
Validation prc-auc = 0.282760
Epoch 285
Validation prc-auc = 0.282822
Epoch 286
Validation prc-auc = 0.282868
Epoch 287
Validation prc-auc = 0.282822
Epoch 288
Validation prc-auc = 0.283000
Epoch 289
Loss = 8.0032e-05, PNorm = 89.2569, GNorm = 0.0035, lr_0 = 6.1671e-04
Validation prc-auc = 0.283046
Epoch 290
Validation prc-auc = 0.282986
Epoch 291
Validation prc-auc = 0.283042
Epoch 292
Validation prc-auc = 0.283172
Epoch 293
Validation prc-auc = 0.283172
Epoch 294
Loss = 7.5541e-05, PNorm = 89.2602, GNorm = 0.0012, lr_0 = 6.1144e-04
Validation prc-auc = 0.283532
Epoch 295
Validation prc-auc = 0.283470
Epoch 296
Validation prc-auc = 0.283409
Epoch 297
Validation prc-auc = 0.283409
Epoch 298
Validation prc-auc = 0.283409
Epoch 299
Loss = 6.5809e-05, PNorm = 89.2681, GNorm = 0.0023, lr_0 = 6.0621e-04
Validation prc-auc = 0.283350
Model 0 best validation prc-auc = 0.490912 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.426883
Ensemble test prc-auc = 0.426883
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.450353
Epoch 1
Validation prc-auc = 0.443115
Epoch 2
Validation prc-auc = 0.453604
Epoch 3
Validation prc-auc = 0.464479
Epoch 4
Loss = 6.7479e-01, PNorm = 78.5519, GNorm = 9.3207, lr_0 = 6.5000e-04
Validation prc-auc = 0.438051
Epoch 5
Validation prc-auc = 0.437230
Epoch 6
Validation prc-auc = 0.488131
Epoch 7
Validation prc-auc = 0.433817
Epoch 8
Validation prc-auc = 0.490877
Epoch 9
Loss = 5.3238e-01, PNorm = 78.7874, GNorm = 4.2839, lr_0 = 9.9743e-04
Validation prc-auc = 0.451423
Epoch 10
Validation prc-auc = 0.447778
Epoch 11
Validation prc-auc = 0.250358
Epoch 12
Validation prc-auc = 0.251147
Epoch 13
Validation prc-auc = 0.293715
Epoch 14
Loss = 5.4735e-01, PNorm = 79.0682, GNorm = 4.1783, lr_0 = 9.8890e-04
Validation prc-auc = 0.430211
Epoch 15
Validation prc-auc = 0.442787
Epoch 16
Validation prc-auc = 0.435221
Epoch 17
Validation prc-auc = 0.442071
Epoch 18
Validation prc-auc = 0.437712
Epoch 19
Loss = 5.9960e-01, PNorm = 79.3356, GNorm = 5.0889, lr_0 = 9.8045e-04
Validation prc-auc = 0.405947
Epoch 20
Validation prc-auc = 0.415426
Epoch 21
Validation prc-auc = 0.398738
Epoch 22
Validation prc-auc = 0.191386
Epoch 23
Validation prc-auc = 0.332591
Epoch 24
Loss = 5.0185e-01, PNorm = 79.6336, GNorm = 1.1886, lr_0 = 9.7207e-04
Validation prc-auc = 0.352359
Epoch 25
Validation prc-auc = 0.346060
Epoch 26
Validation prc-auc = 0.354681
Epoch 27
Validation prc-auc = 0.374514
Epoch 28
Validation prc-auc = 0.359520
Epoch 29
Loss = 3.9279e-01, PNorm = 79.9565, GNorm = 3.0733, lr_0 = 9.6376e-04
Validation prc-auc = 0.394954
Epoch 30
Validation prc-auc = 0.378815
Epoch 31
Validation prc-auc = 0.363490
Epoch 32
Validation prc-auc = 0.284182
Epoch 33
Validation prc-auc = 0.335491
Epoch 34
Loss = 4.0500e-01, PNorm = 80.2851, GNorm = 7.8650, lr_0 = 9.5552e-04
Validation prc-auc = 0.349869
Epoch 35
Validation prc-auc = 0.369388
Epoch 36
Validation prc-auc = 0.391851
Epoch 37
Validation prc-auc = 0.375983
Epoch 38
Validation prc-auc = 0.399716
Epoch 39
Loss = 3.6006e-01, PNorm = 80.6638, GNorm = 2.7468, lr_0 = 9.4735e-04
Validation prc-auc = 0.382586
Epoch 40
Validation prc-auc = 0.383708
Epoch 41
Validation prc-auc = 0.379438
Epoch 42
Validation prc-auc = 0.344584
Epoch 43
Validation prc-auc = 0.356027
Epoch 44
Loss = 3.3708e-01, PNorm = 81.0584, GNorm = 1.1960, lr_0 = 9.3925e-04
Validation prc-auc = 0.356214
Epoch 45
Validation prc-auc = 0.333767
Epoch 46
Validation prc-auc = 0.313628
Epoch 47
Validation prc-auc = 0.336772
Epoch 48
Validation prc-auc = 0.330730
Epoch 49
Loss = 1.9644e-01, PNorm = 81.4938, GNorm = 3.2408, lr_0 = 9.3122e-04
Validation prc-auc = 0.354519
Epoch 50
Validation prc-auc = 0.351713
Epoch 51
Validation prc-auc = 0.354549
Epoch 52
Validation prc-auc = 0.319716
Epoch 53
Validation prc-auc = 0.306216
Epoch 54
Loss = 1.4992e-01, PNorm = 81.9167, GNorm = 0.7446, lr_0 = 9.2326e-04
Validation prc-auc = 0.327580
Epoch 55
Validation prc-auc = 0.325589
Epoch 56
Validation prc-auc = 0.352842
Epoch 57
Validation prc-auc = 0.354549
Epoch 58
Validation prc-auc = 0.348195
Epoch 59
Loss = 1.6586e-01, PNorm = 82.3132, GNorm = 1.7308, lr_0 = 9.1537e-04
Validation prc-auc = 0.305380
Epoch 60
Validation prc-auc = 0.332003
Epoch 61
Validation prc-auc = 0.352183
Epoch 62
Validation prc-auc = 0.269207
Epoch 63
Validation prc-auc = 0.257248
Epoch 64
Loss = 1.8356e-01, PNorm = 82.7234, GNorm = 6.7183, lr_0 = 9.0754e-04
Validation prc-auc = 0.353868
Epoch 65
Validation prc-auc = 0.302649
Epoch 66
Validation prc-auc = 0.318274
Epoch 67
Validation prc-auc = 0.329895
Epoch 68
Validation prc-auc = 0.313174
Epoch 69
Loss = 1.1830e-01, PNorm = 83.1548, GNorm = 0.8571, lr_0 = 8.9978e-04
Validation prc-auc = 0.253649
Epoch 70
Validation prc-auc = 0.259100
Epoch 71
Validation prc-auc = 0.328664
Epoch 72
Validation prc-auc = 0.267724
Epoch 73
Validation prc-auc = 0.286050
Epoch 74
Loss = 6.8920e-02, PNorm = 83.5520, GNorm = 1.7007, lr_0 = 8.9209e-04
Validation prc-auc = 0.352845
Epoch 75
Validation prc-auc = 0.351592
Epoch 76
Validation prc-auc = 0.323956
Epoch 77
Validation prc-auc = 0.292979
Epoch 78
Validation prc-auc = 0.285621
Epoch 79
Loss = 2.5831e-02, PNorm = 83.8893, GNorm = 0.1329, lr_0 = 8.8447e-04
Validation prc-auc = 0.292541
Epoch 80
Validation prc-auc = 0.287925
Epoch 81
Validation prc-auc = 0.297752
Epoch 82
Validation prc-auc = 0.307407
Epoch 83
Validation prc-auc = 0.260515
Epoch 84
Loss = 2.5320e-02, PNorm = 84.1755, GNorm = 0.3427, lr_0 = 8.7691e-04
Validation prc-auc = 0.189323
Epoch 85
Validation prc-auc = 0.225363
Epoch 86
Validation prc-auc = 0.286895
Epoch 87
Validation prc-auc = 0.292944
Epoch 88
Validation prc-auc = 0.292704
Epoch 89
Loss = 2.5646e-01, PNorm = 84.4516, GNorm = 2.0814, lr_0 = 8.6941e-04
Validation prc-auc = 0.360888
Epoch 90
Validation prc-auc = 0.360951
Epoch 91
Validation prc-auc = 0.348303
Epoch 92
Validation prc-auc = 0.320626
Epoch 93
Validation prc-auc = 0.346472
Epoch 94
Loss = 6.0640e-02, PNorm = 84.7099, GNorm = 1.4590, lr_0 = 8.6198e-04
Validation prc-auc = 0.360081
Epoch 95
Validation prc-auc = 0.356844
Epoch 96
Validation prc-auc = 0.342179
Epoch 97
Validation prc-auc = 0.285905
Epoch 98
Validation prc-auc = 0.196533
Epoch 99
Loss = 5.2903e-02, PNorm = 84.9901, GNorm = 0.4358, lr_0 = 8.5461e-04
Validation prc-auc = 0.229233
Epoch 100
Validation prc-auc = 0.249626
Epoch 101
Validation prc-auc = 0.258652
Epoch 102
Validation prc-auc = 0.271943
Epoch 103
Validation prc-auc = 0.290447
Epoch 104
Loss = 2.2823e-02, PNorm = 85.2376, GNorm = 0.5505, lr_0 = 8.4730e-04
Validation prc-auc = 0.321223
Epoch 105
Validation prc-auc = 0.308526
Epoch 106
Validation prc-auc = 0.312397
Epoch 107
Validation prc-auc = 0.292327
Epoch 108
Validation prc-auc = 0.325420
Epoch 109
Loss = 2.6738e-02, PNorm = 85.4408, GNorm = 0.6383, lr_0 = 8.4006e-04
Validation prc-auc = 0.325156
Epoch 110
Validation prc-auc = 0.317045
Epoch 111
Validation prc-auc = 0.318840
Epoch 112
Validation prc-auc = 0.311052
Epoch 113
Validation prc-auc = 0.269121
Epoch 114
Loss = 1.9127e-02, PNorm = 85.6330, GNorm = 1.2177, lr_0 = 8.3288e-04
Validation prc-auc = 0.266165
Epoch 115
Validation prc-auc = 0.249533
Epoch 116
Validation prc-auc = 0.220490
Epoch 117
Validation prc-auc = 0.217333
Epoch 118
Validation prc-auc = 0.242705
Epoch 119
Loss = 2.5329e-03, PNorm = 85.7843, GNorm = 0.0279, lr_0 = 8.2576e-04
Validation prc-auc = 0.251771
Epoch 120
Validation prc-auc = 0.254955
Epoch 121
Validation prc-auc = 0.253266
Epoch 122
Validation prc-auc = 0.236312
Epoch 123
Validation prc-auc = 0.235802
Epoch 124
Loss = 4.2049e-03, PNorm = 85.9025, GNorm = 0.0175, lr_0 = 8.1870e-04
Validation prc-auc = 0.245836
Epoch 125
Validation prc-auc = 0.262948
Epoch 126
Validation prc-auc = 0.274168
Epoch 127
Validation prc-auc = 0.286154
Epoch 128
Validation prc-auc = 0.286708
Epoch 129
Loss = 6.6386e-03, PNorm = 86.0234, GNorm = 0.1598, lr_0 = 8.1170e-04
Validation prc-auc = 0.286670
Epoch 130
Validation prc-auc = 0.280666
Epoch 131
Validation prc-auc = 0.276780
Epoch 132
Validation prc-auc = 0.271997
Epoch 133
Validation prc-auc = 0.271172
Epoch 134
Loss = 2.2648e-03, PNorm = 86.1266, GNorm = 0.0519, lr_0 = 8.0476e-04
Validation prc-auc = 0.273403
Epoch 135
Validation prc-auc = 0.284740
Epoch 136
Validation prc-auc = 0.281803
Epoch 137
Validation prc-auc = 0.269278
Epoch 138
Validation prc-auc = 0.268204
Epoch 139
Loss = 5.9064e-03, PNorm = 86.1971, GNorm = 0.1472, lr_0 = 7.9788e-04
Validation prc-auc = 0.248496
Epoch 140
Validation prc-auc = 0.224663
Epoch 141
Validation prc-auc = 0.237987
Epoch 142
Validation prc-auc = 0.251706
Epoch 143
Validation prc-auc = 0.263135
Epoch 144
Loss = 5.0731e-03, PNorm = 86.2741, GNorm = 0.1962, lr_0 = 7.9106e-04
Validation prc-auc = 0.269497
Epoch 145
Validation prc-auc = 0.288364
Epoch 146
Validation prc-auc = 0.299853
Epoch 147
Validation prc-auc = 0.300752
Epoch 148
Validation prc-auc = 0.307451
Epoch 149
Loss = 4.7443e-03, PNorm = 86.3435, GNorm = 0.1533, lr_0 = 7.8430e-04
Validation prc-auc = 0.308677
Epoch 150
Validation prc-auc = 0.307553
Epoch 151
Validation prc-auc = 0.303300
Epoch 152
Validation prc-auc = 0.301494
Epoch 153
Validation prc-auc = 0.290292
Epoch 154
Loss = 4.1294e-04, PNorm = 86.4086, GNorm = 0.0088, lr_0 = 7.7759e-04
Validation prc-auc = 0.287722
Epoch 155
Validation prc-auc = 0.276216
Epoch 156
Validation prc-auc = 0.275685
Epoch 157
Validation prc-auc = 0.275287
Epoch 158
Validation prc-auc = 0.274977
Epoch 159
Loss = 5.0478e-04, PNorm = 86.4433, GNorm = 0.0198, lr_0 = 7.7095e-04
Validation prc-auc = 0.285617
Epoch 160
Validation prc-auc = 0.291283
Epoch 161
Validation prc-auc = 0.294036
Epoch 162
Validation prc-auc = 0.295108
Epoch 163
Validation prc-auc = 0.294899
Epoch 164
Loss = 1.1425e-03, PNorm = 86.4712, GNorm = 0.0720, lr_0 = 7.6436e-04
Validation prc-auc = 0.303949
Epoch 165
Validation prc-auc = 0.304318
Epoch 166
Validation prc-auc = 0.305851
Epoch 167
Validation prc-auc = 0.308465
Epoch 168
Validation prc-auc = 0.309083
Epoch 169
Loss = 2.4120e-03, PNorm = 86.5002, GNorm = 0.1663, lr_0 = 7.5782e-04
Validation prc-auc = 0.304228
Epoch 170
Validation prc-auc = 0.302654
Epoch 171
Validation prc-auc = 0.300803
Epoch 172
Validation prc-auc = 0.288076
Epoch 173
Validation prc-auc = 0.266056
Epoch 174
Loss = 5.1762e-03, PNorm = 86.5392, GNorm = 0.3394, lr_0 = 7.5134e-04
Validation prc-auc = 0.253052
Epoch 175
Validation prc-auc = 0.249086
Epoch 176
Validation prc-auc = 0.240232
Epoch 177
Validation prc-auc = 0.236760
Epoch 178
Validation prc-auc = 0.245920
Epoch 179
Loss = 4.4986e-04, PNorm = 86.6111, GNorm = 0.0509, lr_0 = 7.4492e-04
Validation prc-auc = 0.255787
Epoch 180
Validation prc-auc = 0.253410
Epoch 181
Validation prc-auc = 0.247341
Epoch 182
Validation prc-auc = 0.251789
Epoch 183
Validation prc-auc = 0.252916
Epoch 184
Loss = 3.4637e-04, PNorm = 86.6513, GNorm = 0.0046, lr_0 = 7.3855e-04
Validation prc-auc = 0.252702
Epoch 185
Validation prc-auc = 0.255836
Epoch 186
Validation prc-auc = 0.261791
Epoch 187
Validation prc-auc = 0.261721
Epoch 188
Validation prc-auc = 0.261903
Epoch 189
Loss = 4.1657e-04, PNorm = 86.6766, GNorm = 0.0157, lr_0 = 7.3224e-04
Validation prc-auc = 0.261880
Epoch 190
Validation prc-auc = 0.261832
Epoch 191
Validation prc-auc = 0.261858
Epoch 192
Validation prc-auc = 0.263395
Epoch 193
Validation prc-auc = 0.263395
Epoch 194
Loss = 3.9351e-04, PNorm = 86.6943, GNorm = 0.0241, lr_0 = 7.2598e-04
Validation prc-auc = 0.263539
Epoch 195
Validation prc-auc = 0.263459
Epoch 196
Validation prc-auc = 0.257308
Epoch 197
Validation prc-auc = 0.257389
Epoch 198
Validation prc-auc = 0.257389
Epoch 199
Loss = 1.3199e-04, PNorm = 86.7039, GNorm = 0.0050, lr_0 = 7.1977e-04
Validation prc-auc = 0.255762
Epoch 200
Validation prc-auc = 0.255898
Epoch 201
Validation prc-auc = 0.255898
Epoch 202
Validation prc-auc = 0.256025
Epoch 203
Validation prc-auc = 0.256097
Epoch 204
Loss = 1.1349e-04, PNorm = 86.7099, GNorm = 0.0032, lr_0 = 7.1362e-04
Validation prc-auc = 0.256097
Epoch 205
Validation prc-auc = 0.256099
Epoch 206
Validation prc-auc = 0.256112
Epoch 207
Validation prc-auc = 0.256198
Epoch 208
Validation prc-auc = 0.249690
Epoch 209
Loss = 8.2639e-05, PNorm = 86.7144, GNorm = 0.0027, lr_0 = 7.0752e-04
Validation prc-auc = 0.249690
Epoch 210
Validation prc-auc = 0.249690
Epoch 211
Validation prc-auc = 0.249690
Epoch 212
Validation prc-auc = 0.249690
Epoch 213
Validation prc-auc = 0.249779
Epoch 214
Loss = 8.4569e-05, PNorm = 86.7185, GNorm = 0.0030, lr_0 = 7.0147e-04
Validation prc-auc = 0.249779
Epoch 215
Validation prc-auc = 0.249762
Epoch 216
Validation prc-auc = 0.249890
Epoch 217
Validation prc-auc = 0.249953
Epoch 218
Validation prc-auc = 0.249953
Epoch 219
Loss = 8.3627e-05, PNorm = 86.7223, GNorm = 0.0024, lr_0 = 6.9548e-04
Validation prc-auc = 0.256533
Epoch 220
Validation prc-auc = 0.256463
Epoch 221
Validation prc-auc = 0.256555
Epoch 222
Validation prc-auc = 0.256555
Epoch 223
Validation prc-auc = 0.257064
Epoch 224
Loss = 7.8145e-05, PNorm = 86.7260, GNorm = 0.0025, lr_0 = 6.8953e-04
Validation prc-auc = 0.257064
Epoch 225
Validation prc-auc = 0.257129
Epoch 226
Validation prc-auc = 0.257129
Epoch 227
Validation prc-auc = 0.257129
Epoch 228
Validation prc-auc = 0.257129
Epoch 229
Loss = 1.1600e-04, PNorm = 86.7296, GNorm = 0.0026, lr_0 = 6.8364e-04
Validation prc-auc = 0.257129
Epoch 230
Validation prc-auc = 0.257129
Epoch 231
Validation prc-auc = 0.257129
Epoch 232
Validation prc-auc = 0.257129
Epoch 233
Validation prc-auc = 0.257129
Epoch 234
Loss = 1.8573e-04, PNorm = 86.7332, GNorm = 0.0031, lr_0 = 6.7779e-04
Validation prc-auc = 0.257065
Epoch 235
Validation prc-auc = 0.257065
Epoch 236
Validation prc-auc = 0.257199
Epoch 237
Validation prc-auc = 0.257277
Epoch 238
Validation prc-auc = 0.257344
Epoch 239
Loss = 9.2596e-05, PNorm = 86.7372, GNorm = 0.0019, lr_0 = 6.7200e-04
Validation prc-auc = 0.257278
Epoch 240
Validation prc-auc = 0.257255
Epoch 241
Validation prc-auc = 0.257255
Epoch 242
Validation prc-auc = 0.257255
Epoch 243
Validation prc-auc = 0.257255
Epoch 244
Loss = 6.1965e-05, PNorm = 86.7413, GNorm = 0.0012, lr_0 = 6.6625e-04
Validation prc-auc = 0.257184
Epoch 245
Validation prc-auc = 0.257184
Epoch 246
Validation prc-auc = 0.257115
Epoch 247
Validation prc-auc = 0.257206
Epoch 248
Validation prc-auc = 0.257206
Epoch 249
Loss = 6.6423e-05, PNorm = 86.7452, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.257139
Epoch 250
Validation prc-auc = 0.257139
Epoch 251
Validation prc-auc = 0.257097
Epoch 252
Validation prc-auc = 0.257097
Epoch 253
Validation prc-auc = 0.257097
Epoch 254
Loss = 4.9712e-05, PNorm = 86.7488, GNorm = 0.0019, lr_0 = 6.5491e-04
Validation prc-auc = 0.257082
Epoch 255
Validation prc-auc = 0.257006
Epoch 256
Validation prc-auc = 0.257082
Epoch 257
Validation prc-auc = 0.257071
Epoch 258
Validation prc-auc = 0.257147
Epoch 259
Loss = 5.1863e-05, PNorm = 86.7524, GNorm = 0.0013, lr_0 = 6.4931e-04
Validation prc-auc = 0.256987
Epoch 260
Validation prc-auc = 0.256963
Epoch 261
Validation prc-auc = 0.256963
Epoch 262
Validation prc-auc = 0.256963
Epoch 263
Validation prc-auc = 0.257030
Epoch 264
Loss = 1.9087e-04, PNorm = 86.7558, GNorm = 0.0013, lr_0 = 6.4376e-04
Validation prc-auc = 0.257163
Epoch 265
Validation prc-auc = 0.257311
Epoch 266
Validation prc-auc = 0.257779
Epoch 267
Validation prc-auc = 0.257779
Epoch 268
Validation prc-auc = 0.257846
Epoch 269
Loss = 7.9699e-05, PNorm = 86.7595, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.263997
Epoch 270
Validation prc-auc = 0.263997
Epoch 271
Validation prc-auc = 0.264088
Epoch 272
Validation prc-auc = 0.264019
Epoch 273
Validation prc-auc = 0.264019
Epoch 274
Loss = 4.5145e-05, PNorm = 86.7629, GNorm = 0.0011, lr_0 = 6.3280e-04
Validation prc-auc = 0.264019
Epoch 275
Validation prc-auc = 0.257869
Epoch 276
Validation prc-auc = 0.264510
Epoch 277
Validation prc-auc = 0.264510
Epoch 278
Validation prc-auc = 0.264443
Epoch 279
Loss = 3.9168e-05, PNorm = 86.7663, GNorm = 0.0010, lr_0 = 6.2739e-04
Validation prc-auc = 0.264443
Epoch 280
Validation prc-auc = 0.264443
Epoch 281
Validation prc-auc = 0.264443
Epoch 282
Validation prc-auc = 0.264537
Epoch 283
Validation prc-auc = 0.264537
Epoch 284
Loss = 1.1023e-04, PNorm = 86.7693, GNorm = 0.0100, lr_0 = 6.2203e-04
Validation prc-auc = 0.264537
Epoch 285
Validation prc-auc = 0.264523
Epoch 286
Validation prc-auc = 0.264590
Epoch 287
Validation prc-auc = 0.264659
Epoch 288
Validation prc-auc = 0.264728
Epoch 289
Loss = 3.2019e-05, PNorm = 86.7725, GNorm = 0.0013, lr_0 = 6.1671e-04
Validation prc-auc = 0.264728
Epoch 290
Validation prc-auc = 0.264728
Epoch 291
Validation prc-auc = 0.264728
Epoch 292
Validation prc-auc = 0.264728
Epoch 293
Validation prc-auc = 0.264718
Epoch 294
Loss = 3.1254e-05, PNorm = 86.7757, GNorm = 0.0010, lr_0 = 6.1144e-04
Validation prc-auc = 0.264718
Epoch 295
Validation prc-auc = 0.264718
Epoch 296
Validation prc-auc = 0.264609
Epoch 297
Validation prc-auc = 0.264609
Epoch 298
Validation prc-auc = 0.264978
Epoch 299
Loss = 3.0260e-05, PNorm = 86.7789, GNorm = 0.0009, lr_0 = 6.0621e-04
Validation prc-auc = 0.264978
Model 0 best validation prc-auc = 0.490877 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387804
Ensemble test prc-auc = 0.387804
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.421469
Epoch 1
Validation prc-auc = 0.423979
Epoch 2
Validation prc-auc = 0.439751
Epoch 3
Validation prc-auc = 0.465896
Epoch 4
Loss = 7.3687e-01, PNorm = 78.5527, GNorm = 8.0153, lr_0 = 6.5000e-04
Validation prc-auc = 0.443333
Epoch 5
Validation prc-auc = 0.463636
Epoch 6
Validation prc-auc = 0.433615
Epoch 7
Validation prc-auc = 0.319327
Epoch 8
Validation prc-auc = 0.433522
Epoch 9
Loss = 5.4549e-01, PNorm = 78.7983, GNorm = 1.2378, lr_0 = 9.9743e-04
Validation prc-auc = 0.439056
Epoch 10
Validation prc-auc = 0.451785
Epoch 11
Validation prc-auc = 0.362315
Epoch 12
Validation prc-auc = 0.331306
Epoch 13
Validation prc-auc = 0.316753
Epoch 14
Loss = 5.2377e-01, PNorm = 79.1024, GNorm = 0.8953, lr_0 = 9.8890e-04
Validation prc-auc = 0.320442
Epoch 15
Validation prc-auc = 0.295246
Epoch 16
Validation prc-auc = 0.325186
Epoch 17
Validation prc-auc = 0.353658
Epoch 18
Validation prc-auc = 0.353847
Epoch 19
Loss = 4.4552e-01, PNorm = 79.4384, GNorm = 1.8933, lr_0 = 9.8045e-04
Validation prc-auc = 0.304821
Epoch 20
Validation prc-auc = 0.304510
Epoch 21
Validation prc-auc = 0.334042
Epoch 22
Validation prc-auc = 0.388249
Epoch 23
Validation prc-auc = 0.385059
Epoch 24
Loss = 3.6079e-01, PNorm = 79.7877, GNorm = 2.4063, lr_0 = 9.7207e-04
Validation prc-auc = 0.456953
Epoch 25
Validation prc-auc = 0.445516
Epoch 26
Validation prc-auc = 0.298666
Epoch 27
Validation prc-auc = 0.266809
Epoch 28
Validation prc-auc = 0.268973
Epoch 29
Loss = 3.4658e-01, PNorm = 80.1289, GNorm = 0.8745, lr_0 = 9.6376e-04
Validation prc-auc = 0.318488
Epoch 30
Validation prc-auc = 0.326917
Epoch 31
Validation prc-auc = 0.329532
Epoch 32
Validation prc-auc = 0.325767
Epoch 33
Validation prc-auc = 0.166782
Epoch 34
Loss = 2.9142e-01, PNorm = 80.4997, GNorm = 1.4784, lr_0 = 9.5552e-04
Validation prc-auc = 0.263719
Epoch 35
Validation prc-auc = 0.281547
Epoch 36
Validation prc-auc = 0.323984
Epoch 37
Validation prc-auc = 0.341547
Epoch 38
Validation prc-auc = 0.352572
Epoch 39
Loss = 3.0176e-01, PNorm = 80.9185, GNorm = 1.8169, lr_0 = 9.4735e-04
Validation prc-auc = 0.345094
Epoch 40
Validation prc-auc = 0.307132
Epoch 41
Validation prc-auc = 0.325801
Epoch 42
Validation prc-auc = 0.341158
Epoch 43
Validation prc-auc = 0.332618
Epoch 44
Loss = 1.5395e-01, PNorm = 81.3193, GNorm = 1.4727, lr_0 = 9.3925e-04
Validation prc-auc = 0.346512
Epoch 45
Validation prc-auc = 0.314487
Epoch 46
Validation prc-auc = 0.321420
Epoch 47
Validation prc-auc = 0.331676
Epoch 48
Validation prc-auc = 0.325380
Epoch 49
Loss = 8.4553e-02, PNorm = 81.7248, GNorm = 0.4701, lr_0 = 9.3122e-04
Validation prc-auc = 0.336953
Epoch 50
Validation prc-auc = 0.317782
Epoch 51
Validation prc-auc = 0.334362
Epoch 52
Validation prc-auc = 0.344895
Epoch 53
Validation prc-auc = 0.338960
Epoch 54
Loss = 1.1898e-01, PNorm = 82.1275, GNorm = 1.7424, lr_0 = 9.2326e-04
Validation prc-auc = 0.312097
Epoch 55
Validation prc-auc = 0.224829
Epoch 56
Validation prc-auc = 0.211702
Epoch 57
Validation prc-auc = 0.228625
Epoch 58
Validation prc-auc = 0.249902
Epoch 59
Loss = 1.3664e-01, PNorm = 82.5256, GNorm = 0.8003, lr_0 = 9.1537e-04
Validation prc-auc = 0.315017
Epoch 60
Validation prc-auc = 0.171745
Epoch 61
Validation prc-auc = 0.254731
Epoch 62
Validation prc-auc = 0.270841
Epoch 63
Validation prc-auc = 0.216812
Epoch 64
Loss = 2.4207e-01, PNorm = 82.9790, GNorm = 3.0624, lr_0 = 9.0754e-04
Validation prc-auc = 0.303712
Epoch 65
Validation prc-auc = 0.300287
Epoch 66
Validation prc-auc = 0.227536
Epoch 67
Validation prc-auc = 0.234641
Epoch 68
Validation prc-auc = 0.293496
Epoch 69
Loss = 4.8540e-02, PNorm = 83.3779, GNorm = 0.2424, lr_0 = 8.9978e-04
Validation prc-auc = 0.277111
Epoch 70
Validation prc-auc = 0.254592
Epoch 71
Validation prc-auc = 0.258545
Epoch 72
Validation prc-auc = 0.237237
Epoch 73
Validation prc-auc = 0.219126
Epoch 74
Loss = 1.3329e-01, PNorm = 83.6973, GNorm = 2.4291, lr_0 = 8.9209e-04
Validation prc-auc = 0.237532
Epoch 75
Validation prc-auc = 0.266272
Epoch 76
Validation prc-auc = 0.282868
Epoch 77
Validation prc-auc = 0.245745
Epoch 78
Validation prc-auc = 0.281975
Epoch 79
Loss = 3.0196e-02, PNorm = 83.9751, GNorm = 0.2708, lr_0 = 8.8447e-04
Validation prc-auc = 0.328357
Epoch 80
Validation prc-auc = 0.321928
Epoch 81
Validation prc-auc = 0.235448
Epoch 82
Validation prc-auc = 0.195459
Epoch 83
Validation prc-auc = 0.192653
Epoch 84
Loss = 2.9629e-02, PNorm = 84.2209, GNorm = 0.1373, lr_0 = 8.7691e-04
Validation prc-auc = 0.239309
Epoch 85
Validation prc-auc = 0.304257
Epoch 86
Validation prc-auc = 0.333307
Epoch 87
Validation prc-auc = 0.289372
Epoch 88
Validation prc-auc = 0.236526
Epoch 89
Loss = 2.6510e-02, PNorm = 84.4304, GNorm = 0.0461, lr_0 = 8.6941e-04
Validation prc-auc = 0.198907
Epoch 90
Validation prc-auc = 0.251381
Epoch 91
Validation prc-auc = 0.278850
Epoch 92
Validation prc-auc = 0.279741
Epoch 93
Validation prc-auc = 0.248908
Epoch 94
Loss = 4.4098e-02, PNorm = 84.6117, GNorm = 0.1094, lr_0 = 8.6198e-04
Validation prc-auc = 0.217561
Epoch 95
Validation prc-auc = 0.220587
Epoch 96
Validation prc-auc = 0.230401
Epoch 97
Validation prc-auc = 0.225088
Epoch 98
Validation prc-auc = 0.193584
Epoch 99
Loss = 1.4977e-02, PNorm = 84.7991, GNorm = 0.3501, lr_0 = 8.5461e-04
Validation prc-auc = 0.176868
Epoch 100
Validation prc-auc = 0.170880
Epoch 101
Validation prc-auc = 0.172012
Epoch 102
Validation prc-auc = 0.185316
Epoch 103
Validation prc-auc = 0.199213
Epoch 104
Loss = 5.0745e-03, PNorm = 84.9753, GNorm = 0.2658, lr_0 = 8.4730e-04
Validation prc-auc = 0.211431
Epoch 105
Validation prc-auc = 0.217693
Epoch 106
Validation prc-auc = 0.230831
Epoch 107
Validation prc-auc = 0.240978
Epoch 108
Validation prc-auc = 0.263327
Epoch 109
Loss = 1.2869e-02, PNorm = 85.1049, GNorm = 1.0465, lr_0 = 8.4006e-04
Validation prc-auc = 0.267876
Epoch 110
Validation prc-auc = 0.234446
Epoch 111
Validation prc-auc = 0.221977
Epoch 112
Validation prc-auc = 0.209011
Epoch 113
Validation prc-auc = 0.207965
Epoch 114
Loss = 2.2331e-03, PNorm = 85.1923, GNorm = 0.1035, lr_0 = 8.3288e-04
Validation prc-auc = 0.216497
Epoch 115
Validation prc-auc = 0.226325
Epoch 116
Validation prc-auc = 0.227724
Epoch 117
Validation prc-auc = 0.235186
Epoch 118
Validation prc-auc = 0.237335
Epoch 119
Loss = 2.6992e-02, PNorm = 85.2644, GNorm = 0.3377, lr_0 = 8.2576e-04
Validation prc-auc = 0.266233
Epoch 120
Validation prc-auc = 0.261343
Epoch 121
Validation prc-auc = 0.233877
Epoch 122
Validation prc-auc = 0.229014
Epoch 123
Validation prc-auc = 0.238096
Epoch 124
Loss = 6.5969e-02, PNorm = 85.3855, GNorm = 1.1910, lr_0 = 8.1870e-04
Validation prc-auc = 0.269577
Epoch 125
Validation prc-auc = 0.276810
Epoch 126
Validation prc-auc = 0.269922
Epoch 127
Validation prc-auc = 0.243705
Epoch 128
Validation prc-auc = 0.224063
Epoch 129
Loss = 1.1894e-02, PNorm = 85.5518, GNorm = 0.3324, lr_0 = 8.1170e-04
Validation prc-auc = 0.202577
Epoch 130
Validation prc-auc = 0.213873
Epoch 131
Validation prc-auc = 0.218412
Epoch 132
Validation prc-auc = 0.245407
Epoch 133
Validation prc-auc = 0.270409
Epoch 134
Loss = 1.1480e-02, PNorm = 85.7105, GNorm = 0.5125, lr_0 = 8.0476e-04
Validation prc-auc = 0.270824
Epoch 135
Validation prc-auc = 0.270837
Epoch 136
Validation prc-auc = 0.268830
Epoch 137
Validation prc-auc = 0.260741
Epoch 138
Validation prc-auc = 0.263618
Epoch 139
Loss = 1.1048e-02, PNorm = 85.8041, GNorm = 0.5878, lr_0 = 7.9788e-04
Validation prc-auc = 0.224960
Epoch 140
Validation prc-auc = 0.222092
Epoch 141
Validation prc-auc = 0.220771
Epoch 142
Validation prc-auc = 0.220701
Epoch 143
Validation prc-auc = 0.214179
Epoch 144
Loss = 1.4845e-03, PNorm = 85.9090, GNorm = 0.1000, lr_0 = 7.9106e-04
Validation prc-auc = 0.215867
Epoch 145
Validation prc-auc = 0.214265
Epoch 146
Validation prc-auc = 0.216375
Epoch 147
Validation prc-auc = 0.213805
Epoch 148
Validation prc-auc = 0.214617
Epoch 149
Loss = 1.1243e-03, PNorm = 85.9787, GNorm = 0.0499, lr_0 = 7.8430e-04
Validation prc-auc = 0.212677
Epoch 150
Validation prc-auc = 0.217421
Epoch 151
Validation prc-auc = 0.225611
Epoch 152
Validation prc-auc = 0.226140
Epoch 153
Validation prc-auc = 0.226447
Epoch 154
Loss = 6.7331e-04, PNorm = 86.0258, GNorm = 0.0165, lr_0 = 7.7759e-04
Validation prc-auc = 0.225323
Epoch 155
Validation prc-auc = 0.228061
Epoch 156
Validation prc-auc = 0.228166
Epoch 157
Validation prc-auc = 0.231046
Epoch 158
Validation prc-auc = 0.230086
Epoch 159
Loss = 5.1876e-04, PNorm = 86.0613, GNorm = 0.0073, lr_0 = 7.7095e-04
Validation prc-auc = 0.230086
Epoch 160
Validation prc-auc = 0.231706
Epoch 161
Validation prc-auc = 0.230949
Epoch 162
Validation prc-auc = 0.226674
Epoch 163
Validation prc-auc = 0.224061
Epoch 164
Loss = 5.7099e-04, PNorm = 86.0859, GNorm = 0.0521, lr_0 = 7.6436e-04
Validation prc-auc = 0.229333
Epoch 165
Validation prc-auc = 0.235687
Epoch 166
Validation prc-auc = 0.235814
Epoch 167
Validation prc-auc = 0.236074
Epoch 168
Validation prc-auc = 0.235992
Epoch 169
Loss = 5.0852e-04, PNorm = 86.1070, GNorm = 0.0154, lr_0 = 7.5782e-04
Validation prc-auc = 0.235922
Epoch 170
Validation prc-auc = 0.232089
Epoch 171
Validation prc-auc = 0.231757
Epoch 172
Validation prc-auc = 0.231473
Epoch 173
Validation prc-auc = 0.231533
Epoch 174
Loss = 5.2526e-04, PNorm = 86.1244, GNorm = 0.0538, lr_0 = 7.5134e-04
Validation prc-auc = 0.231316
Epoch 175
Validation prc-auc = 0.231248
Epoch 176
Validation prc-auc = 0.231120
Epoch 177
Validation prc-auc = 0.230419
Epoch 178
Validation prc-auc = 0.230486
Epoch 179
Loss = 2.8719e-04, PNorm = 86.1390, GNorm = 0.0099, lr_0 = 7.4492e-04
Validation prc-auc = 0.230486
Epoch 180
Validation prc-auc = 0.230486
Epoch 181
Validation prc-auc = 0.230697
Epoch 182
Validation prc-auc = 0.230764
Epoch 183
Validation prc-auc = 0.230314
Epoch 184
Loss = 1.7340e-04, PNorm = 86.1509, GNorm = 0.0039, lr_0 = 7.3855e-04
Validation prc-auc = 0.230314
Epoch 185
Validation prc-auc = 0.234203
Epoch 186
Validation prc-auc = 0.234203
Epoch 187
Validation prc-auc = 0.234422
Epoch 188
Validation prc-auc = 0.234355
Epoch 189
Loss = 1.6721e-04, PNorm = 86.1599, GNorm = 0.0051, lr_0 = 7.3224e-04
Validation prc-auc = 0.234355
Epoch 190
Validation prc-auc = 0.234288
Epoch 191
Validation prc-auc = 0.238891
Epoch 192
Validation prc-auc = 0.238958
Epoch 193
Validation prc-auc = 0.238749
Epoch 194
Loss = 2.0894e-04, PNorm = 86.1678, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.238749
Epoch 195
Validation prc-auc = 0.238815
Epoch 196
Validation prc-auc = 0.238585
Epoch 197
Validation prc-auc = 0.238585
Epoch 198
Validation prc-auc = 0.238706
Epoch 199
Loss = 7.1296e-05, PNorm = 86.1752, GNorm = 0.0019, lr_0 = 7.1977e-04
Validation prc-auc = 0.240320
Epoch 200
Validation prc-auc = 0.240320
Epoch 201
Validation prc-auc = 0.240320
Epoch 202
Validation prc-auc = 0.240320
Epoch 203
Validation prc-auc = 0.240179
Epoch 204
Loss = 2.1907e-04, PNorm = 86.1820, GNorm = 0.0056, lr_0 = 7.1362e-04
Validation prc-auc = 0.239934
Epoch 205
Validation prc-auc = 0.240002
Epoch 206
Validation prc-auc = 0.239845
Epoch 207
Validation prc-auc = 0.239845
Epoch 208
Validation prc-auc = 0.239710
Epoch 209
Loss = 5.3842e-05, PNorm = 86.1887, GNorm = 0.0024, lr_0 = 7.0752e-04
Validation prc-auc = 0.239772
Epoch 210
Validation prc-auc = 0.239772
Epoch 211
Validation prc-auc = 0.239898
Epoch 212
Validation prc-auc = 0.239898
Epoch 213
Validation prc-auc = 0.239965
Epoch 214
Loss = 1.1752e-04, PNorm = 86.1954, GNorm = 0.0012, lr_0 = 7.0147e-04
Validation prc-auc = 0.240030
Epoch 215
Validation prc-auc = 0.240170
Epoch 216
Validation prc-auc = 0.240849
Epoch 217
Validation prc-auc = 0.240921
Epoch 218
Validation prc-auc = 0.241056
Epoch 219
Loss = 8.7771e-05, PNorm = 86.2030, GNorm = 0.0032, lr_0 = 6.9548e-04
Validation prc-auc = 0.244760
Epoch 220
Validation prc-auc = 0.244760
Epoch 221
Validation prc-auc = 0.244690
Epoch 222
Validation prc-auc = 0.244626
Epoch 223
Validation prc-auc = 0.244837
Epoch 224
Loss = 7.8586e-05, PNorm = 86.2105, GNorm = 0.0031, lr_0 = 6.8953e-04
Validation prc-auc = 0.244524
Epoch 225
Validation prc-auc = 0.244362
Epoch 226
Validation prc-auc = 0.249124
Epoch 227
Validation prc-auc = 0.249188
Epoch 228
Validation prc-auc = 0.249326
Epoch 229
Loss = 5.3647e-05, PNorm = 86.2180, GNorm = 0.0025, lr_0 = 6.8364e-04
Validation prc-auc = 0.249394
Epoch 230
Validation prc-auc = 0.249394
Epoch 231
Validation prc-auc = 0.249323
Epoch 232
Validation prc-auc = 0.249323
Epoch 233
Validation prc-auc = 0.249774
Epoch 234
Loss = 4.6062e-05, PNorm = 86.2253, GNorm = 0.0011, lr_0 = 6.7779e-04
Validation prc-auc = 0.249422
Epoch 235
Validation prc-auc = 0.249270
Epoch 236
Validation prc-auc = 0.249270
Epoch 237
Validation prc-auc = 0.249123
Epoch 238
Validation prc-auc = 0.249323
Epoch 239
Loss = 4.2064e-05, PNorm = 86.2315, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.249323
Epoch 240
Validation prc-auc = 0.249186
Epoch 241
Validation prc-auc = 0.248821
Epoch 242
Validation prc-auc = 0.249550
Epoch 243
Validation prc-auc = 0.249271
Epoch 244
Loss = 5.0311e-05, PNorm = 86.2382, GNorm = 0.0018, lr_0 = 6.6625e-04
Validation prc-auc = 0.249340
Epoch 245
Validation prc-auc = 0.249273
Epoch 246
Validation prc-auc = 0.249273
Epoch 247
Validation prc-auc = 0.249343
Epoch 248
Validation prc-auc = 0.249351
Epoch 249
Loss = 2.3754e-05, PNorm = 86.2447, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.249420
Epoch 250
Validation prc-auc = 0.249700
Epoch 251
Validation prc-auc = 0.250617
Epoch 252
Validation prc-auc = 0.250836
Epoch 253
Validation prc-auc = 0.254120
Epoch 254
Loss = 1.9915e-05, PNorm = 86.2511, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.253981
Epoch 255
Validation prc-auc = 0.252300
Epoch 256
Validation prc-auc = 0.252157
Epoch 257
Validation prc-auc = 0.251569
Epoch 258
Validation prc-auc = 0.251691
Epoch 259
Loss = 1.3458e-04, PNorm = 86.2563, GNorm = 0.0071, lr_0 = 6.4931e-04
Validation prc-auc = 0.254110
Epoch 260
Validation prc-auc = 0.254779
Epoch 261
Validation prc-auc = 0.258094
Epoch 262
Validation prc-auc = 0.258455
Epoch 263
Validation prc-auc = 0.258931
Epoch 264
Loss = 2.1682e-05, PNorm = 86.2614, GNorm = 0.0007, lr_0 = 6.4376e-04
Validation prc-auc = 0.258857
Epoch 265
Validation prc-auc = 0.258299
Epoch 266
Validation prc-auc = 0.257925
Epoch 267
Validation prc-auc = 0.251979
Epoch 268
Validation prc-auc = 0.252051
Epoch 269
Loss = 1.4931e-05, PNorm = 86.2663, GNorm = 0.0010, lr_0 = 6.3826e-04
Validation prc-auc = 0.251905
Epoch 270
Validation prc-auc = 0.251832
Epoch 271
Validation prc-auc = 0.251908
Epoch 272
Validation prc-auc = 0.254670
Epoch 273
Validation prc-auc = 0.254670
Epoch 274
Loss = 1.6231e-05, PNorm = 86.2704, GNorm = 0.0012, lr_0 = 6.3280e-04
Validation prc-auc = 0.254670
Epoch 275
Validation prc-auc = 0.254597
Epoch 276
Validation prc-auc = 0.254526
Epoch 277
Validation prc-auc = 0.258474
Epoch 278
Validation prc-auc = 0.262857
Epoch 279
Loss = 1.1756e-05, PNorm = 86.2735, GNorm = 0.0004, lr_0 = 6.2739e-04
Validation prc-auc = 0.262866
Epoch 280
Validation prc-auc = 0.259768
Epoch 281
Validation prc-auc = 0.260306
Epoch 282
Validation prc-auc = 0.266757
Epoch 283
Validation prc-auc = 0.264661
Epoch 284
Loss = 1.2980e-05, PNorm = 86.2767, GNorm = 0.0006, lr_0 = 6.2203e-04
Validation prc-auc = 0.264400
Epoch 285
Validation prc-auc = 0.265007
Epoch 286
Validation prc-auc = 0.268412
Epoch 287
Validation prc-auc = 0.267190
Epoch 288
Validation prc-auc = 0.266879
Epoch 289
Loss = 7.5815e-06, PNorm = 86.2803, GNorm = 0.0003, lr_0 = 6.1671e-04
Validation prc-auc = 0.263334
Epoch 290
Validation prc-auc = 0.260106
Epoch 291
Validation prc-auc = 0.263176
Epoch 292
Validation prc-auc = 0.266582
Epoch 293
Validation prc-auc = 0.266857
Epoch 294
Loss = 2.3784e-05, PNorm = 86.2831, GNorm = 0.0008, lr_0 = 6.1144e-04
Validation prc-auc = 0.270101
Epoch 295
Validation prc-auc = 0.274080
Epoch 296
Validation prc-auc = 0.274169
Epoch 297
Validation prc-auc = 0.273711
Epoch 298
Validation prc-auc = 0.273629
Epoch 299
Loss = 1.2231e-05, PNorm = 86.2869, GNorm = 0.0006, lr_0 = 6.0621e-04
Validation prc-auc = 0.273753
Model 0 best validation prc-auc = 0.465896 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.381926
Ensemble test prc-auc = 0.381926
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.435708
Epoch 1
Validation prc-auc = 0.441723
Epoch 2
Validation prc-auc = 0.454004
Epoch 3
Validation prc-auc = 0.425858
Epoch 4
Loss = 8.8239e-01, PNorm = 78.5518, GNorm = 11.6588, lr_0 = 6.5000e-04
Validation prc-auc = 0.422802
Epoch 5
Validation prc-auc = 0.446360
Epoch 6
Validation prc-auc = 0.497502
Epoch 7
Validation prc-auc = 0.450476
Epoch 8
Validation prc-auc = 0.460662
Epoch 9
Loss = 5.7273e-01, PNorm = 78.8056, GNorm = 6.4887, lr_0 = 9.9743e-04
Validation prc-auc = 0.499893
Epoch 10
Validation prc-auc = 0.474553
Epoch 11
Validation prc-auc = 0.500709
Epoch 12
Validation prc-auc = 0.465800
Epoch 13
Validation prc-auc = 0.422143
Epoch 14
Loss = 4.7215e-01, PNorm = 79.0723, GNorm = 0.8683, lr_0 = 9.8890e-04
Validation prc-auc = 0.437083
Epoch 15
Validation prc-auc = 0.427480
Epoch 16
Validation prc-auc = 0.303268
Epoch 17
Validation prc-auc = 0.290890
Epoch 18
Validation prc-auc = 0.290655
Epoch 19
Loss = 4.8168e-01, PNorm = 79.3140, GNorm = 3.5564, lr_0 = 9.8045e-04
Validation prc-auc = 0.310672
Epoch 20
Validation prc-auc = 0.382304
Epoch 21
Validation prc-auc = 0.399580
Epoch 22
Validation prc-auc = 0.486670
Epoch 23
Validation prc-auc = 0.478546
Epoch 24
Loss = 3.9170e-01, PNorm = 79.5683, GNorm = 3.1018, lr_0 = 9.7207e-04
Validation prc-auc = 0.485464
Epoch 25
Validation prc-auc = 0.476910
Epoch 26
Validation prc-auc = 0.524092
Epoch 27
Validation prc-auc = 0.502251
Epoch 28
Validation prc-auc = 0.455351
Epoch 29
Loss = 4.1243e-01, PNorm = 79.8351, GNorm = 4.8494, lr_0 = 9.6376e-04
Validation prc-auc = 0.429718
Epoch 30
Validation prc-auc = 0.390266
Epoch 31
Validation prc-auc = 0.278632
Epoch 32
Validation prc-auc = 0.292380
Epoch 33
Validation prc-auc = 0.277029
Epoch 34
Loss = 3.6675e-01, PNorm = 80.1210, GNorm = 1.2056, lr_0 = 9.5552e-04
Validation prc-auc = 0.275304
Epoch 35
Validation prc-auc = 0.303900
Epoch 36
Validation prc-auc = 0.271541
Epoch 37
Validation prc-auc = 0.298444
Epoch 38
Validation prc-auc = 0.299185
Epoch 39
Loss = 2.2963e-01, PNorm = 80.4620, GNorm = 2.7669, lr_0 = 9.4735e-04
Validation prc-auc = 0.307856
Epoch 40
Validation prc-auc = 0.325169
Epoch 41
Validation prc-auc = 0.332641
Epoch 42
Validation prc-auc = 0.330515
Epoch 43
Validation prc-auc = 0.345459
Epoch 44
Loss = 2.7264e-01, PNorm = 80.7752, GNorm = 3.9265, lr_0 = 9.3925e-04
Validation prc-auc = 0.325186
Epoch 45
Validation prc-auc = 0.303068
Epoch 46
Validation prc-auc = 0.229226
Epoch 47
Validation prc-auc = 0.296760
Epoch 48
Validation prc-auc = 0.261483
Epoch 49
Loss = 1.7006e-01, PNorm = 81.1059, GNorm = 1.2697, lr_0 = 9.3122e-04
Validation prc-auc = 0.292475
Epoch 50
Validation prc-auc = 0.330260
Epoch 51
Validation prc-auc = 0.218820
Epoch 52
Validation prc-auc = 0.273329
Epoch 53
Validation prc-auc = 0.242914
Epoch 54
Loss = 1.9472e-01, PNorm = 81.4353, GNorm = 2.1499, lr_0 = 9.2326e-04
Validation prc-auc = 0.160746
Epoch 55
Validation prc-auc = 0.272838
Epoch 56
Validation prc-auc = 0.338849
Epoch 57
Validation prc-auc = 0.333858
Epoch 58
Validation prc-auc = 0.228223
Epoch 59
Loss = 1.6898e-01, PNorm = 81.7784, GNorm = 1.6197, lr_0 = 9.1537e-04
Validation prc-auc = 0.205948
Epoch 60
Validation prc-auc = 0.240472
Epoch 61
Validation prc-auc = 0.278798
Epoch 62
Validation prc-auc = 0.212128
Epoch 63
Validation prc-auc = 0.317435
Epoch 64
Loss = 8.3831e-02, PNorm = 82.1227, GNorm = 1.7327, lr_0 = 9.0754e-04
Validation prc-auc = 0.353449
Epoch 65
Validation prc-auc = 0.340085
Epoch 66
Validation prc-auc = 0.318553
Epoch 67
Validation prc-auc = 0.204701
Epoch 68
Validation prc-auc = 0.195564
Epoch 69
Loss = 9.3059e-02, PNorm = 82.4415, GNorm = 0.6657, lr_0 = 8.9978e-04
Validation prc-auc = 0.233861
Epoch 70
Validation prc-auc = 0.260961
Epoch 71
Validation prc-auc = 0.266063
Epoch 72
Validation prc-auc = 0.252819
Epoch 73
Validation prc-auc = 0.293240
Epoch 74
Loss = 3.4455e-02, PNorm = 82.7423, GNorm = 0.8372, lr_0 = 8.9209e-04
Validation prc-auc = 0.244053
Epoch 75
Validation prc-auc = 0.246577
Epoch 76
Validation prc-auc = 0.280393
Epoch 77
Validation prc-auc = 0.281936
Epoch 78
Validation prc-auc = 0.299902
Epoch 79
Loss = 6.6255e-02, PNorm = 83.0010, GNorm = 0.5624, lr_0 = 8.8447e-04
Validation prc-auc = 0.290115
Epoch 80
Validation prc-auc = 0.287320
Epoch 81
Validation prc-auc = 0.240154
Epoch 82
Validation prc-auc = 0.213205
Epoch 83
Validation prc-auc = 0.240780
Epoch 84
Loss = 5.5168e-02, PNorm = 83.2307, GNorm = 1.3664, lr_0 = 8.7691e-04
Validation prc-auc = 0.233837
Epoch 85
Validation prc-auc = 0.207928
Epoch 86
Validation prc-auc = 0.197484
Epoch 87
Validation prc-auc = 0.212902
Epoch 88
Validation prc-auc = 0.221800
Epoch 89
Loss = 2.5878e-02, PNorm = 83.4308, GNorm = 0.4328, lr_0 = 8.6941e-04
Validation prc-auc = 0.265031
Epoch 90
Validation prc-auc = 0.231669
Epoch 91
Validation prc-auc = 0.237319
Epoch 92
Validation prc-auc = 0.284283
Epoch 93
Validation prc-auc = 0.304206
Epoch 94
Loss = 1.6962e-02, PNorm = 83.6489, GNorm = 1.1727, lr_0 = 8.6198e-04
Validation prc-auc = 0.343293
Epoch 95
Validation prc-auc = 0.345032
Epoch 96
Validation prc-auc = 0.342256
Epoch 97
Validation prc-auc = 0.291128
Epoch 98
Validation prc-auc = 0.210684
Epoch 99
Loss = 1.9196e-02, PNorm = 83.8497, GNorm = 0.4856, lr_0 = 8.5461e-04
Validation prc-auc = 0.192574
Epoch 100
Validation prc-auc = 0.209601
Epoch 101
Validation prc-auc = 0.250479
Epoch 102
Validation prc-auc = 0.278383
Epoch 103
Validation prc-auc = 0.275290
Epoch 104
Loss = 7.1857e-03, PNorm = 84.0357, GNorm = 0.2142, lr_0 = 8.4730e-04
Validation prc-auc = 0.264635
Epoch 105
Validation prc-auc = 0.254282
Epoch 106
Validation prc-auc = 0.264551
Epoch 107
Validation prc-auc = 0.294027
Epoch 108
Validation prc-auc = 0.286518
Epoch 109
Loss = 9.0730e-03, PNorm = 84.1814, GNorm = 0.8205, lr_0 = 8.4006e-04
Validation prc-auc = 0.270109
Epoch 110
Validation prc-auc = 0.268162
Epoch 111
Validation prc-auc = 0.259521
Epoch 112
Validation prc-auc = 0.205277
Epoch 113
Validation prc-auc = 0.166000
Epoch 114
Loss = 4.9406e-02, PNorm = 84.3463, GNorm = 1.9243, lr_0 = 8.3288e-04
Validation prc-auc = 0.212214
Epoch 115
Validation prc-auc = 0.263161
Epoch 116
Validation prc-auc = 0.279538
Epoch 117
Validation prc-auc = 0.219397
Epoch 118
Validation prc-auc = 0.253764
Epoch 119
Loss = 2.6383e-02, PNorm = 84.5484, GNorm = 0.2824, lr_0 = 8.2576e-04
Validation prc-auc = 0.273885
Epoch 120
Validation prc-auc = 0.246262
Epoch 121
Validation prc-auc = 0.191422
Epoch 122
Validation prc-auc = 0.216522
Epoch 123
Validation prc-auc = 0.219966
Epoch 124
Loss = 6.1804e-02, PNorm = 84.7972, GNorm = 1.0151, lr_0 = 8.1870e-04
Validation prc-auc = 0.185787
Epoch 125
Validation prc-auc = 0.200373
Epoch 126
Validation prc-auc = 0.242191
Epoch 127
Validation prc-auc = 0.240296
Epoch 128
Validation prc-auc = 0.178176
Epoch 129
Loss = 3.3523e-02, PNorm = 85.1395, GNorm = 0.5256, lr_0 = 8.1170e-04
Validation prc-auc = 0.204151
Epoch 130
Validation prc-auc = 0.218663
Epoch 131
Validation prc-auc = 0.236722
Epoch 132
Validation prc-auc = 0.215811
Epoch 133
Validation prc-auc = 0.202910
Epoch 134
Loss = 2.1253e-02, PNorm = 85.4255, GNorm = 0.4560, lr_0 = 8.0476e-04
Validation prc-auc = 0.196777
Epoch 135
Validation prc-auc = 0.205130
Epoch 136
Validation prc-auc = 0.227541
Epoch 137
Validation prc-auc = 0.252211
Epoch 138
Validation prc-auc = 0.245700
Epoch 139
Loss = 1.3113e-02, PNorm = 85.6583, GNorm = 0.0394, lr_0 = 7.9788e-04
Validation prc-auc = 0.222978
Epoch 140
Validation prc-auc = 0.207730
Epoch 141
Validation prc-auc = 0.198344
Epoch 142
Validation prc-auc = 0.196818
Epoch 143
Validation prc-auc = 0.201621
Epoch 144
Loss = 9.6422e-03, PNorm = 85.8139, GNorm = 0.3940, lr_0 = 7.9106e-04
Validation prc-auc = 0.215057
Epoch 145
Validation prc-auc = 0.227365
Epoch 146
Validation prc-auc = 0.252091
Epoch 147
Validation prc-auc = 0.275157
Epoch 148
Validation prc-auc = 0.289513
Epoch 149
Loss = 7.7328e-03, PNorm = 85.9599, GNorm = 0.0524, lr_0 = 7.8430e-04
Validation prc-auc = 0.274051
Epoch 150
Validation prc-auc = 0.253683
Epoch 151
Validation prc-auc = 0.198872
Epoch 152
Validation prc-auc = 0.189283
Epoch 153
Validation prc-auc = 0.191910
Epoch 154
Loss = 7.6692e-02, PNorm = 86.0930, GNorm = 1.6271, lr_0 = 7.7759e-04
Validation prc-auc = 0.228123
Epoch 155
Validation prc-auc = 0.272988
Epoch 156
Validation prc-auc = 0.286564
Epoch 157
Validation prc-auc = 0.280947
Epoch 158
Validation prc-auc = 0.238652
Epoch 159
Loss = 1.5534e-03, PNorm = 86.2253, GNorm = 0.0538, lr_0 = 7.7095e-04
Validation prc-auc = 0.203787
Epoch 160
Validation prc-auc = 0.174199
Epoch 161
Validation prc-auc = 0.165425
Epoch 162
Validation prc-auc = 0.167142
Epoch 163
Validation prc-auc = 0.173436
Epoch 164
Loss = 5.2604e-03, PNorm = 86.3228, GNorm = 0.0879, lr_0 = 7.6436e-04
Validation prc-auc = 0.182415
Epoch 165
Validation prc-auc = 0.194889
Epoch 166
Validation prc-auc = 0.218257
Epoch 167
Validation prc-auc = 0.238327
Epoch 168
Validation prc-auc = 0.247811
Epoch 169
Loss = 6.8510e-03, PNorm = 86.4010, GNorm = 0.3714, lr_0 = 7.5782e-04
Validation prc-auc = 0.245389
Epoch 170
Validation prc-auc = 0.243590
Epoch 171
Validation prc-auc = 0.238518
Epoch 172
Validation prc-auc = 0.241509
Epoch 173
Validation prc-auc = 0.232566
Epoch 174
Loss = 6.0077e-04, PNorm = 86.4580, GNorm = 0.0183, lr_0 = 7.5134e-04
Validation prc-auc = 0.228940
Epoch 175
Validation prc-auc = 0.230387
Epoch 176
Validation prc-auc = 0.217262
Epoch 177
Validation prc-auc = 0.226027
Epoch 178
Validation prc-auc = 0.234933
Epoch 179
Loss = 1.2234e-02, PNorm = 86.5019, GNorm = 0.3012, lr_0 = 7.4492e-04
Validation prc-auc = 0.237254
Epoch 180
Validation prc-auc = 0.231102
Epoch 181
Validation prc-auc = 0.231997
Epoch 182
Validation prc-auc = 0.225636
Epoch 183
Validation prc-auc = 0.225364
Epoch 184
Loss = 1.1444e-03, PNorm = 86.5539, GNorm = 0.0869, lr_0 = 7.3855e-04
Validation prc-auc = 0.225363
Epoch 185
Validation prc-auc = 0.220254
Epoch 186
Validation prc-auc = 0.226408
Epoch 187
Validation prc-auc = 0.230151
Epoch 188
Validation prc-auc = 0.243914
Epoch 189
Loss = 1.5982e-03, PNorm = 86.5929, GNorm = 0.0528, lr_0 = 7.3224e-04
Validation prc-auc = 0.244414
Epoch 190
Validation prc-auc = 0.243082
Epoch 191
Validation prc-auc = 0.243138
Epoch 192
Validation prc-auc = 0.255176
Epoch 193
Validation prc-auc = 0.255786
Epoch 194
Loss = 7.6292e-04, PNorm = 86.6298, GNorm = 0.0353, lr_0 = 7.2598e-04
Validation prc-auc = 0.257413
Epoch 195
Validation prc-auc = 0.245336
Epoch 196
Validation prc-auc = 0.245865
Epoch 197
Validation prc-auc = 0.245849
Epoch 198
Validation prc-auc = 0.251351
Epoch 199
Loss = 4.9669e-04, PNorm = 86.6535, GNorm = 0.0209, lr_0 = 7.1977e-04
Validation prc-auc = 0.251486
Epoch 200
Validation prc-auc = 0.252000
Epoch 201
Validation prc-auc = 0.246061
Epoch 202
Validation prc-auc = 0.246128
Epoch 203
Validation prc-auc = 0.246128
Epoch 204
Loss = 2.3363e-04, PNorm = 86.6683, GNorm = 0.0039, lr_0 = 7.1362e-04
Validation prc-auc = 0.246311
Epoch 205
Validation prc-auc = 0.246404
Epoch 206
Validation prc-auc = 0.246159
Epoch 207
Validation prc-auc = 0.246159
Epoch 208
Validation prc-auc = 0.246227
Epoch 209
Loss = 3.2975e-04, PNorm = 86.6775, GNorm = 0.0150, lr_0 = 7.0752e-04
Validation prc-auc = 0.245827
Epoch 210
Validation prc-auc = 0.243924
Epoch 211
Validation prc-auc = 0.243597
Epoch 212
Validation prc-auc = 0.243450
Epoch 213
Validation prc-auc = 0.243279
Epoch 214
Loss = 3.8837e-04, PNorm = 86.6858, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.238650
Epoch 215
Validation prc-auc = 0.238392
Epoch 216
Validation prc-auc = 0.238233
Epoch 217
Validation prc-auc = 0.237587
Epoch 218
Validation prc-auc = 0.237032
Epoch 219
Loss = 4.7102e-04, PNorm = 86.6975, GNorm = 0.0065, lr_0 = 6.9548e-04
Validation prc-auc = 0.236958
Epoch 220
Validation prc-auc = 0.236481
Epoch 221
Validation prc-auc = 0.236329
Epoch 222
Validation prc-auc = 0.236706
Epoch 223
Validation prc-auc = 0.236978
Epoch 224
Loss = 1.0850e-04, PNorm = 86.7101, GNorm = 0.0023, lr_0 = 6.8953e-04
Validation prc-auc = 0.241532
Epoch 225
Validation prc-auc = 0.241425
Epoch 226
Validation prc-auc = 0.241691
Epoch 227
Validation prc-auc = 0.241903
Epoch 228
Validation prc-auc = 0.241592
Epoch 229
Loss = 1.9321e-04, PNorm = 86.7197, GNorm = 0.0052, lr_0 = 6.8364e-04
Validation prc-auc = 0.241513
Epoch 230
Validation prc-auc = 0.241515
Epoch 231
Validation prc-auc = 0.241367
Epoch 232
Validation prc-auc = 0.240913
Epoch 233
Validation prc-auc = 0.241020
Epoch 234
Loss = 1.1889e-04, PNorm = 86.7272, GNorm = 0.0030, lr_0 = 6.7779e-04
Validation prc-auc = 0.241020
Epoch 235
Validation prc-auc = 0.241082
Epoch 236
Validation prc-auc = 0.240905
Epoch 237
Validation prc-auc = 0.240910
Epoch 238
Validation prc-auc = 0.240509
Epoch 239
Loss = 1.9137e-04, PNorm = 86.7349, GNorm = 0.0056, lr_0 = 6.7200e-04
Validation prc-auc = 0.240506
Epoch 240
Validation prc-auc = 0.240262
Epoch 241
Validation prc-auc = 0.240089
Epoch 242
Validation prc-auc = 0.240016
Epoch 243
Validation prc-auc = 0.240016
Epoch 244
Loss = 1.2605e-04, PNorm = 86.7433, GNorm = 0.0025, lr_0 = 6.6625e-04
Validation prc-auc = 0.246301
Epoch 245
Validation prc-auc = 0.246373
Epoch 246
Validation prc-auc = 0.240089
Epoch 247
Validation prc-auc = 0.240089
Epoch 248
Validation prc-auc = 0.239977
Epoch 249
Loss = 1.0577e-04, PNorm = 86.7513, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.240178
Epoch 250
Validation prc-auc = 0.240178
Epoch 251
Validation prc-auc = 0.240176
Epoch 252
Validation prc-auc = 0.239800
Epoch 253
Validation prc-auc = 0.240341
Epoch 254
Loss = 4.3154e-05, PNorm = 86.7612, GNorm = 0.0018, lr_0 = 6.5491e-04
Validation prc-auc = 0.240416
Epoch 255
Validation prc-auc = 0.240238
Epoch 256
Validation prc-auc = 0.240495
Epoch 257
Validation prc-auc = 0.240490
Epoch 258
Validation prc-auc = 0.240557
Epoch 259
Loss = 5.0080e-05, PNorm = 86.7691, GNorm = 0.0008, lr_0 = 6.4931e-04
Validation prc-auc = 0.240635
Epoch 260
Validation prc-auc = 0.240310
Epoch 261
Validation prc-auc = 0.240373
Epoch 262
Validation prc-auc = 0.240293
Epoch 263
Validation prc-auc = 0.240440
Epoch 264
Loss = 5.8278e-05, PNorm = 86.7764, GNorm = 0.0018, lr_0 = 6.4376e-04
Validation prc-auc = 0.240860
Epoch 265
Validation prc-auc = 0.241244
Epoch 266
Validation prc-auc = 0.241244
Epoch 267
Validation prc-auc = 0.241372
Epoch 268
Validation prc-auc = 0.241298
Epoch 269
Loss = 1.0803e-04, PNorm = 86.7852, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.241465
Epoch 270
Validation prc-auc = 0.241605
Epoch 271
Validation prc-auc = 0.241605
Epoch 272
Validation prc-auc = 0.241691
Epoch 273
Validation prc-auc = 0.241691
Epoch 274
Loss = 1.1218e-04, PNorm = 86.7927, GNorm = 0.0035, lr_0 = 6.3280e-04
Validation prc-auc = 0.241691
Epoch 275
Validation prc-auc = 0.243121
Epoch 276
Validation prc-auc = 0.243121
Epoch 277
Validation prc-auc = 0.242998
Epoch 278
Validation prc-auc = 0.242833
Epoch 279
Loss = 1.2764e-04, PNorm = 86.8023, GNorm = 0.0025, lr_0 = 6.2739e-04
Validation prc-auc = 0.243086
Epoch 280
Validation prc-auc = 0.243345
Epoch 281
Validation prc-auc = 0.243438
Epoch 282
Validation prc-auc = 0.243802
Epoch 283
Validation prc-auc = 0.243802
Epoch 284
Loss = 1.4733e-04, PNorm = 86.8134, GNorm = 0.0133, lr_0 = 6.2203e-04
Validation prc-auc = 0.243856
Epoch 285
Validation prc-auc = 0.243856
Epoch 286
Validation prc-auc = 0.244168
Epoch 287
Validation prc-auc = 0.244098
Epoch 288
Validation prc-auc = 0.244098
Epoch 289
Loss = 3.4529e-05, PNorm = 86.8239, GNorm = 0.0006, lr_0 = 6.1671e-04
Validation prc-auc = 0.243966
Epoch 290
Validation prc-auc = 0.243693
Epoch 291
Validation prc-auc = 0.243623
Epoch 292
Validation prc-auc = 0.243940
Epoch 293
Validation prc-auc = 0.244010
Epoch 294
Loss = 3.3168e-05, PNorm = 86.8315, GNorm = 0.0009, lr_0 = 6.1144e-04
Validation prc-auc = 0.250264
Epoch 295
Validation prc-auc = 0.249965
Epoch 296
Validation prc-auc = 0.249854
Epoch 297
Validation prc-auc = 0.249821
Epoch 298
Validation prc-auc = 0.249514
Epoch 299
Loss = 3.6395e-05, PNorm = 86.8380, GNorm = 0.0027, lr_0 = 6.0621e-04
Validation prc-auc = 0.249688
Model 0 best validation prc-auc = 0.524092 on epoch 26
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.512151
Ensemble test prc-auc = 0.512151
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.451981
Epoch 1
Validation prc-auc = 0.446994
Epoch 2
Validation prc-auc = 0.456972
Epoch 3
Validation prc-auc = 0.434877
Epoch 4
Loss = 7.3300e-01, PNorm = 78.5513, GNorm = 15.6126, lr_0 = 6.5000e-04
Validation prc-auc = 0.447210
Epoch 5
Validation prc-auc = 0.363943
Epoch 6
Validation prc-auc = 0.448088
Epoch 7
Validation prc-auc = 0.492841
Epoch 8
Validation prc-auc = 0.466519
Epoch 9
Loss = 6.7786e-01, PNorm = 78.8123, GNorm = 6.3473, lr_0 = 9.9743e-04
Validation prc-auc = 0.485870
Epoch 10
Validation prc-auc = 0.319707
Epoch 11
Validation prc-auc = 0.311744
Epoch 12
Validation prc-auc = 0.295678
Epoch 13
Validation prc-auc = 0.431364
Epoch 14
Loss = 5.5333e-01, PNorm = 79.1009, GNorm = 3.2639, lr_0 = 9.8890e-04
Validation prc-auc = 0.291894
Epoch 15
Validation prc-auc = 0.298993
Epoch 16
Validation prc-auc = 0.320662
Epoch 17
Validation prc-auc = 0.412705
Epoch 18
Validation prc-auc = 0.392252
Epoch 19
Loss = 4.9499e-01, PNorm = 79.3513, GNorm = 1.1391, lr_0 = 9.8045e-04
Validation prc-auc = 0.398266
Epoch 20
Validation prc-auc = 0.415484
Epoch 21
Validation prc-auc = 0.302287
Epoch 22
Validation prc-auc = 0.241544
Epoch 23
Validation prc-auc = 0.332738
Epoch 24
Loss = 4.7328e-01, PNorm = 79.6106, GNorm = 4.8199, lr_0 = 9.7207e-04
Validation prc-auc = 0.361985
Epoch 25
Validation prc-auc = 0.338897
Epoch 26
Validation prc-auc = 0.362187
Epoch 27
Validation prc-auc = 0.343060
Epoch 28
Validation prc-auc = 0.315752
Epoch 29
Loss = 4.4372e-01, PNorm = 79.9035, GNorm = 5.2715, lr_0 = 9.6376e-04
Validation prc-auc = 0.294318
Epoch 30
Validation prc-auc = 0.390667
Epoch 31
Validation prc-auc = 0.411278
Epoch 32
Validation prc-auc = 0.419696
Epoch 33
Validation prc-auc = 0.427415
Epoch 34
Loss = 4.3009e-01, PNorm = 80.2022, GNorm = 3.1468, lr_0 = 9.5552e-04
Validation prc-auc = 0.421484
Epoch 35
Validation prc-auc = 0.332690
Epoch 36
Validation prc-auc = 0.343598
Epoch 37
Validation prc-auc = 0.348775
Epoch 38
Validation prc-auc = 0.382581
Epoch 39
Loss = 3.1726e-01, PNorm = 80.4962, GNorm = 0.8414, lr_0 = 9.4735e-04
Validation prc-auc = 0.381942
Epoch 40
Validation prc-auc = 0.362318
Epoch 41
Validation prc-auc = 0.355381
Epoch 42
Validation prc-auc = 0.322578
Epoch 43
Validation prc-auc = 0.332220
Epoch 44
Loss = 2.5246e-01, PNorm = 80.8630, GNorm = 1.3613, lr_0 = 9.3925e-04
Validation prc-auc = 0.316774
Epoch 45
Validation prc-auc = 0.297024
Epoch 46
Validation prc-auc = 0.283733
Epoch 47
Validation prc-auc = 0.291580
Epoch 48
Validation prc-auc = 0.286660
Epoch 49
Loss = 1.8035e-01, PNorm = 81.2976, GNorm = 1.0920, lr_0 = 9.3122e-04
Validation prc-auc = 0.272602
Epoch 50
Validation prc-auc = 0.287387
Epoch 51
Validation prc-auc = 0.266252
Epoch 52
Validation prc-auc = 0.163727
Epoch 53
Validation prc-auc = 0.213129
Epoch 54
Loss = 1.4254e-01, PNorm = 81.7439, GNorm = 0.5898, lr_0 = 9.2326e-04
Validation prc-auc = 0.231894
Epoch 55
Validation prc-auc = 0.380404
Epoch 56
Validation prc-auc = 0.350888
Epoch 57
Validation prc-auc = 0.358191
Epoch 58
Validation prc-auc = 0.363654
Epoch 59
Loss = 1.2802e-01, PNorm = 82.1729, GNorm = 1.3726, lr_0 = 9.1537e-04
Validation prc-auc = 0.296677
Epoch 60
Validation prc-auc = 0.280951
Epoch 61
Validation prc-auc = 0.272732
Epoch 62
Validation prc-auc = 0.256215
Epoch 63
Validation prc-auc = 0.261775
Epoch 64
Loss = 6.7330e-02, PNorm = 82.5714, GNorm = 1.0194, lr_0 = 9.0754e-04
Validation prc-auc = 0.266497
Epoch 65
Validation prc-auc = 0.302952
Epoch 66
Validation prc-auc = 0.313754
Epoch 67
Validation prc-auc = 0.313195
Epoch 68
Validation prc-auc = 0.331574
Epoch 69
Loss = 9.0878e-02, PNorm = 82.9257, GNorm = 2.1943, lr_0 = 8.9978e-04
Validation prc-auc = 0.300863
Epoch 70
Validation prc-auc = 0.286141
Epoch 71
Validation prc-auc = 0.296993
Epoch 72
Validation prc-auc = 0.281545
Epoch 73
Validation prc-auc = 0.276575
Epoch 74
Loss = 2.3740e-02, PNorm = 83.2514, GNorm = 0.3164, lr_0 = 8.9209e-04
Validation prc-auc = 0.282224
Epoch 75
Validation prc-auc = 0.279330
Epoch 76
Validation prc-auc = 0.263277
Epoch 77
Validation prc-auc = 0.255019
Epoch 78
Validation prc-auc = 0.271515
Epoch 79
Loss = 4.1753e-02, PNorm = 83.5512, GNorm = 0.4995, lr_0 = 8.8447e-04
Validation prc-auc = 0.315346
Epoch 80
Validation prc-auc = 0.330518
Epoch 81
Validation prc-auc = 0.306628
Epoch 82
Validation prc-auc = 0.302877
Epoch 83
Validation prc-auc = 0.333172
Epoch 84
Loss = 1.0037e-02, PNorm = 83.8114, GNorm = 0.1544, lr_0 = 8.7691e-04
Validation prc-auc = 0.332299
Epoch 85
Validation prc-auc = 0.334477
Epoch 86
Validation prc-auc = 0.329965
Epoch 87
Validation prc-auc = 0.270618
Epoch 88
Validation prc-auc = 0.278125
Epoch 89
Loss = 1.0980e-01, PNorm = 84.0138, GNorm = 1.2087, lr_0 = 8.6941e-04
Validation prc-auc = 0.307675
Epoch 90
Validation prc-auc = 0.331740
Epoch 91
Validation prc-auc = 0.351743
Epoch 92
Validation prc-auc = 0.350187
Epoch 93
Validation prc-auc = 0.318731
Epoch 94
Loss = 1.5660e-01, PNorm = 84.2226, GNorm = 1.2927, lr_0 = 8.6198e-04
Validation prc-auc = 0.326551
Epoch 95
Validation prc-auc = 0.291725
Epoch 96
Validation prc-auc = 0.195668
Epoch 97
Validation prc-auc = 0.184707
Epoch 98
Validation prc-auc = 0.232739
Epoch 99
Loss = 6.2176e-02, PNorm = 84.4634, GNorm = 2.4517, lr_0 = 8.5461e-04
Validation prc-auc = 0.248480
Epoch 100
Validation prc-auc = 0.179112
Epoch 101
Validation prc-auc = 0.201404
Epoch 102
Validation prc-auc = 0.247289
Epoch 103
Validation prc-auc = 0.251840
Epoch 104
Loss = 3.0495e-02, PNorm = 84.7787, GNorm = 2.0164, lr_0 = 8.4730e-04
Validation prc-auc = 0.284531
Epoch 105
Validation prc-auc = 0.336537
Epoch 106
Validation prc-auc = 0.356780
Epoch 107
Validation prc-auc = 0.359496
Epoch 108
Validation prc-auc = 0.332561
Epoch 109
Loss = 2.3595e-02, PNorm = 85.1265, GNorm = 0.2145, lr_0 = 8.4006e-04
Validation prc-auc = 0.289069
Epoch 110
Validation prc-auc = 0.262988
Epoch 111
Validation prc-auc = 0.271154
Epoch 112
Validation prc-auc = 0.264520
Epoch 113
Validation prc-auc = 0.269497
Epoch 114
Loss = 2.2038e-01, PNorm = 85.4287, GNorm = 2.4526, lr_0 = 8.3288e-04
Validation prc-auc = 0.244269
Epoch 115
Validation prc-auc = 0.184818
Epoch 116
Validation prc-auc = 0.165806
Epoch 117
Validation prc-auc = 0.254500
Epoch 118
Validation prc-auc = 0.312299
Epoch 119
Loss = 5.2687e-02, PNorm = 85.7742, GNorm = 0.7315, lr_0 = 8.2576e-04
Validation prc-auc = 0.274522
Epoch 120
Validation prc-auc = 0.226121
Epoch 121
Validation prc-auc = 0.215974
Epoch 122
Validation prc-auc = 0.204843
Epoch 123
Validation prc-auc = 0.243778
Epoch 124
Loss = 6.1667e-02, PNorm = 86.0668, GNorm = 2.7278, lr_0 = 8.1870e-04
Validation prc-auc = 0.215740
Epoch 125
Validation prc-auc = 0.204278
Epoch 126
Validation prc-auc = 0.237765
Epoch 127
Validation prc-auc = 0.236781
Epoch 128
Validation prc-auc = 0.202069
Epoch 129
Loss = 3.9290e-02, PNorm = 86.4271, GNorm = 1.4355, lr_0 = 8.1170e-04
Validation prc-auc = 0.171611
Epoch 130
Validation prc-auc = 0.188763
Epoch 131
Validation prc-auc = 0.273622
Epoch 132
Validation prc-auc = 0.285016
Epoch 133
Validation prc-auc = 0.295705
Epoch 134
Loss = 2.2487e-02, PNorm = 86.7805, GNorm = 0.6677, lr_0 = 8.0476e-04
Validation prc-auc = 0.295033
Epoch 135
Validation prc-auc = 0.313064
Epoch 136
Validation prc-auc = 0.314268
Epoch 137
Validation prc-auc = 0.305511
Epoch 138
Validation prc-auc = 0.300970
Epoch 139
Loss = 1.4917e-02, PNorm = 87.0262, GNorm = 0.8027, lr_0 = 7.9788e-04
Validation prc-auc = 0.289984
Epoch 140
Validation prc-auc = 0.289688
Epoch 141
Validation prc-auc = 0.272827
Epoch 142
Validation prc-auc = 0.243509
Epoch 143
Validation prc-auc = 0.229837
Epoch 144
Loss = 1.1503e-02, PNorm = 87.2025, GNorm = 0.1404, lr_0 = 7.9106e-04
Validation prc-auc = 0.200323
Epoch 145
Validation prc-auc = 0.179604
Epoch 146
Validation prc-auc = 0.177672
Epoch 147
Validation prc-auc = 0.190052
Epoch 148
Validation prc-auc = 0.203944
Epoch 149
Loss = 1.1095e-02, PNorm = 87.3288, GNorm = 0.3025, lr_0 = 7.8430e-04
Validation prc-auc = 0.204917
Epoch 150
Validation prc-auc = 0.209018
Epoch 151
Validation prc-auc = 0.209511
Epoch 152
Validation prc-auc = 0.204461
Epoch 153
Validation prc-auc = 0.237384
Epoch 154
Loss = 3.4870e-03, PNorm = 87.4270, GNorm = 0.1493, lr_0 = 7.7759e-04
Validation prc-auc = 0.246913
Epoch 155
Validation prc-auc = 0.250661
Epoch 156
Validation prc-auc = 0.259806
Epoch 157
Validation prc-auc = 0.263446
Epoch 158
Validation prc-auc = 0.269533
Epoch 159
Loss = 2.8702e-03, PNorm = 87.5061, GNorm = 0.0384, lr_0 = 7.7095e-04
Validation prc-auc = 0.274294
Epoch 160
Validation prc-auc = 0.261164
Epoch 161
Validation prc-auc = 0.260509
Epoch 162
Validation prc-auc = 0.257529
Epoch 163
Validation prc-auc = 0.237820
Epoch 164
Loss = 2.2486e-03, PNorm = 87.5652, GNorm = 0.0561, lr_0 = 7.6436e-04
Validation prc-auc = 0.234714
Epoch 165
Validation prc-auc = 0.223052
Epoch 166
Validation prc-auc = 0.230105
Epoch 167
Validation prc-auc = 0.230004
Epoch 168
Validation prc-auc = 0.226148
Epoch 169
Loss = 1.4940e-03, PNorm = 87.6145, GNorm = 0.0345, lr_0 = 7.5782e-04
Validation prc-auc = 0.225737
Epoch 170
Validation prc-auc = 0.217345
Epoch 171
Validation prc-auc = 0.217235
Epoch 172
Validation prc-auc = 0.217558
Epoch 173
Validation prc-auc = 0.223173
Epoch 174
Loss = 8.7847e-04, PNorm = 87.6570, GNorm = 0.0279, lr_0 = 7.5134e-04
Validation prc-auc = 0.222471
Epoch 175
Validation prc-auc = 0.222174
Epoch 176
Validation prc-auc = 0.220916
Epoch 177
Validation prc-auc = 0.220301
Epoch 178
Validation prc-auc = 0.220193
Epoch 179
Loss = 4.9069e-04, PNorm = 87.6854, GNorm = 0.0094, lr_0 = 7.4492e-04
Validation prc-auc = 0.219958
Epoch 180
Validation prc-auc = 0.219895
Epoch 181
Validation prc-auc = 0.221900
Epoch 182
Validation prc-auc = 0.222981
Epoch 183
Validation prc-auc = 0.231372
Epoch 184
Loss = 1.1542e-03, PNorm = 87.7007, GNorm = 0.0182, lr_0 = 7.3855e-04
Validation prc-auc = 0.233313
Epoch 185
Validation prc-auc = 0.240918
Epoch 186
Validation prc-auc = 0.242466
Epoch 187
Validation prc-auc = 0.242526
Epoch 188
Validation prc-auc = 0.244635
Epoch 189
Loss = 2.0666e-03, PNorm = 87.7300, GNorm = 0.0243, lr_0 = 7.3224e-04
Validation prc-auc = 0.252570
Epoch 190
Validation prc-auc = 0.265322
Epoch 191
Validation prc-auc = 0.261996
Epoch 192
Validation prc-auc = 0.264496
Epoch 193
Validation prc-auc = 0.255808
Epoch 194
Loss = 4.4080e-03, PNorm = 87.7632, GNorm = 0.4909, lr_0 = 7.2598e-04
Validation prc-auc = 0.243308
Epoch 195
Validation prc-auc = 0.249994
Epoch 196
Validation prc-auc = 0.238423
Epoch 197
Validation prc-auc = 0.234646
Epoch 198
Validation prc-auc = 0.247178
Epoch 199
Loss = 7.2347e-04, PNorm = 87.7936, GNorm = 0.0180, lr_0 = 7.1977e-04
Validation prc-auc = 0.246259
Epoch 200
Validation prc-auc = 0.251045
Epoch 201
Validation prc-auc = 0.257668
Epoch 202
Validation prc-auc = 0.257949
Epoch 203
Validation prc-auc = 0.260627
Epoch 204
Loss = 7.0931e-04, PNorm = 87.8198, GNorm = 0.0246, lr_0 = 7.1362e-04
Validation prc-auc = 0.261541
Epoch 205
Validation prc-auc = 0.261455
Epoch 206
Validation prc-auc = 0.264477
Epoch 207
Validation prc-auc = 0.262961
Epoch 208
Validation prc-auc = 0.263147
Epoch 209
Loss = 4.5923e-04, PNorm = 87.8424, GNorm = 0.0146, lr_0 = 7.0752e-04
Validation prc-auc = 0.260400
Epoch 210
Validation prc-auc = 0.254651
Epoch 211
Validation prc-auc = 0.254685
Epoch 212
Validation prc-auc = 0.254610
Epoch 213
Validation prc-auc = 0.255038
Epoch 214
Loss = 3.3965e-04, PNorm = 87.8602, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.255038
Epoch 215
Validation prc-auc = 0.254281
Epoch 216
Validation prc-auc = 0.253146
Epoch 217
Validation prc-auc = 0.253506
Epoch 218
Validation prc-auc = 0.254100
Epoch 219
Loss = 6.8978e-04, PNorm = 87.8762, GNorm = 0.0085, lr_0 = 6.9548e-04
Validation prc-auc = 0.252629
Epoch 220
Validation prc-auc = 0.261809
Epoch 221
Validation prc-auc = 0.268167
Epoch 222
Validation prc-auc = 0.268249
Epoch 223
Validation prc-auc = 0.268976
Epoch 224
Loss = 4.5019e-04, PNorm = 87.8891, GNorm = 0.0198, lr_0 = 6.8953e-04
Validation prc-auc = 0.269516
Epoch 225
Validation prc-auc = 0.274278
Epoch 226
Validation prc-auc = 0.274109
Epoch 227
Validation prc-auc = 0.273516
Epoch 228
Validation prc-auc = 0.265833
Epoch 229
Loss = 1.7040e-04, PNorm = 87.9016, GNorm = 0.0061, lr_0 = 6.8364e-04
Validation prc-auc = 0.270684
Epoch 230
Validation prc-auc = 0.270185
Epoch 231
Validation prc-auc = 0.270694
Epoch 232
Validation prc-auc = 0.270751
Epoch 233
Validation prc-auc = 0.270631
Epoch 234
Loss = 1.4315e-04, PNorm = 87.9119, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.270455
Epoch 235
Validation prc-auc = 0.270370
Epoch 236
Validation prc-auc = 0.270231
Epoch 237
Validation prc-auc = 0.270231
Epoch 238
Validation prc-auc = 0.270231
Epoch 239
Loss = 1.9387e-04, PNorm = 87.9192, GNorm = 0.0094, lr_0 = 6.7200e-04
Validation prc-auc = 0.269676
Epoch 240
Validation prc-auc = 0.269793
Epoch 241
Validation prc-auc = 0.269793
Epoch 242
Validation prc-auc = 0.269793
Epoch 243
Validation prc-auc = 0.269793
Epoch 244
Loss = 1.6600e-04, PNorm = 87.9262, GNorm = 0.0035, lr_0 = 6.6625e-04
Validation prc-auc = 0.269894
Epoch 245
Validation prc-auc = 0.270105
Epoch 246
Validation prc-auc = 0.276827
Epoch 247
Validation prc-auc = 0.276363
Epoch 248
Validation prc-auc = 0.276339
Epoch 249
Loss = 1.9239e-04, PNorm = 87.9332, GNorm = 0.0101, lr_0 = 6.6056e-04
Validation prc-auc = 0.275668
Epoch 250
Validation prc-auc = 0.275034
Epoch 251
Validation prc-auc = 0.268477
Epoch 252
Validation prc-auc = 0.268418
Epoch 253
Validation prc-auc = 0.268747
Epoch 254
Loss = 1.3926e-04, PNorm = 87.9405, GNorm = 0.0027, lr_0 = 6.5491e-04
Validation prc-auc = 0.268457
Epoch 255
Validation prc-auc = 0.268238
Epoch 256
Validation prc-auc = 0.268125
Epoch 257
Validation prc-auc = 0.268185
Epoch 258
Validation prc-auc = 0.275545
Epoch 259
Loss = 9.2985e-05, PNorm = 87.9478, GNorm = 0.0034, lr_0 = 6.4931e-04
Validation prc-auc = 0.275510
Epoch 260
Validation prc-auc = 0.275650
Epoch 261
Validation prc-auc = 0.276198
Epoch 262
Validation prc-auc = 0.276275
Epoch 263
Validation prc-auc = 0.276275
Epoch 264
Loss = 1.4267e-04, PNorm = 87.9554, GNorm = 0.0052, lr_0 = 6.4376e-04
Validation prc-auc = 0.276192
Epoch 265
Validation prc-auc = 0.276192
Epoch 266
Validation prc-auc = 0.276054
Epoch 267
Validation prc-auc = 0.276054
Epoch 268
Validation prc-auc = 0.276232
Epoch 269
Loss = 1.1702e-04, PNorm = 87.9637, GNorm = 0.0053, lr_0 = 6.3826e-04
Validation prc-auc = 0.276206
Epoch 270
Validation prc-auc = 0.276298
Epoch 271
Validation prc-auc = 0.276481
Epoch 272
Validation prc-auc = 0.276481
Epoch 273
Validation prc-auc = 0.276481
Epoch 274
Loss = 3.3947e-05, PNorm = 87.9706, GNorm = 0.0006, lr_0 = 6.3280e-04
Validation prc-auc = 0.276481
Epoch 275
Validation prc-auc = 0.276337
Epoch 276
Validation prc-auc = 0.276512
Epoch 277
Validation prc-auc = 0.276575
Epoch 278
Validation prc-auc = 0.275938
Epoch 279
Loss = 6.3084e-05, PNorm = 87.9764, GNorm = 0.0055, lr_0 = 6.2739e-04
Validation prc-auc = 0.275856
Epoch 280
Validation prc-auc = 0.275670
Epoch 281
Validation prc-auc = 0.275638
Epoch 282
Validation prc-auc = 0.264682
Epoch 283
Validation prc-auc = 0.265450
Epoch 284
Loss = 7.0341e-05, PNorm = 87.9810, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.265515
Epoch 285
Validation prc-auc = 0.265515
Epoch 286
Validation prc-auc = 0.265436
Epoch 287
Validation prc-auc = 0.275830
Epoch 288
Validation prc-auc = 0.275823
Epoch 289
Loss = 4.6313e-05, PNorm = 87.9878, GNorm = 0.0019, lr_0 = 6.1671e-04
Validation prc-auc = 0.275288
Epoch 290
Validation prc-auc = 0.274733
Epoch 291
Validation prc-auc = 0.274733
Epoch 292
Validation prc-auc = 0.274351
Epoch 293
Validation prc-auc = 0.274579
Epoch 294
Loss = 1.4626e-04, PNorm = 87.9938, GNorm = 0.0015, lr_0 = 6.1144e-04
Validation prc-auc = 0.274670
Epoch 295
Validation prc-auc = 0.274853
Epoch 296
Validation prc-auc = 0.275408
Epoch 297
Validation prc-auc = 0.276237
Epoch 298
Validation prc-auc = 0.277788
Epoch 299
Loss = 4.2091e-05, PNorm = 87.9985, GNorm = 0.0032, lr_0 = 6.0621e-04
Validation prc-auc = 0.277967
Model 0 best validation prc-auc = 0.492841 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.437878
Ensemble test prc-auc = 0.437878
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.447435
Epoch 1
Validation prc-auc = 0.455436
Epoch 2
Validation prc-auc = 0.437530
Epoch 3
Validation prc-auc = 0.433102
Epoch 4
Loss = 6.7931e-01, PNorm = 78.5478, GNorm = 5.8820, lr_0 = 6.5000e-04
Validation prc-auc = 0.440586
Epoch 5
Validation prc-auc = 0.497692
Epoch 6
Validation prc-auc = 0.424765
Epoch 7
Validation prc-auc = 0.395213
Epoch 8
Validation prc-auc = 0.294183
Epoch 9
Loss = 5.2405e-01, PNorm = 78.7933, GNorm = 6.3021, lr_0 = 9.9743e-04
Validation prc-auc = 0.335632
Epoch 10
Validation prc-auc = 0.283611
Epoch 11
Validation prc-auc = 0.289981
Epoch 12
Validation prc-auc = 0.306816
Epoch 13
Validation prc-auc = 0.280692
Epoch 14
Loss = 5.3578e-01, PNorm = 79.0958, GNorm = 5.6050, lr_0 = 9.8890e-04
Validation prc-auc = 0.455066
Epoch 15
Validation prc-auc = 0.476875
Epoch 16
Validation prc-auc = 0.488546
Epoch 17
Validation prc-auc = 0.270603
Epoch 18
Validation prc-auc = 0.275675
Epoch 19
Loss = 5.5998e-01, PNorm = 79.4052, GNorm = 11.0964, lr_0 = 9.8045e-04
Validation prc-auc = 0.270895
Epoch 20
Validation prc-auc = 0.401254
Epoch 21
Validation prc-auc = 0.414864
Epoch 22
Validation prc-auc = 0.423959
Epoch 23
Validation prc-auc = 0.415277
Epoch 24
Loss = 4.8958e-01, PNorm = 79.7725, GNorm = 2.0160, lr_0 = 9.7207e-04
Validation prc-auc = 0.295220
Epoch 25
Validation prc-auc = 0.323758
Epoch 26
Validation prc-auc = 0.364850
Epoch 27
Validation prc-auc = 0.338250
Epoch 28
Validation prc-auc = 0.334354
Epoch 29
Loss = 3.8850e-01, PNorm = 80.1224, GNorm = 3.6337, lr_0 = 9.6376e-04
Validation prc-auc = 0.319221
Epoch 30
Validation prc-auc = 0.292483
Epoch 31
Validation prc-auc = 0.276862
Epoch 32
Validation prc-auc = 0.329416
Epoch 33
Validation prc-auc = 0.317479
Epoch 34
Loss = 2.8719e-01, PNorm = 80.4957, GNorm = 1.0380, lr_0 = 9.5552e-04
Validation prc-auc = 0.358420
Epoch 35
Validation prc-auc = 0.317927
Epoch 36
Validation prc-auc = 0.296096
Epoch 37
Validation prc-auc = 0.275368
Epoch 38
Validation prc-auc = 0.245756
Epoch 39
Loss = 3.0297e-01, PNorm = 80.8953, GNorm = 1.1152, lr_0 = 9.4735e-04
Validation prc-auc = 0.261501
Epoch 40
Validation prc-auc = 0.315310
Epoch 41
Validation prc-auc = 0.302531
Epoch 42
Validation prc-auc = 0.318977
Epoch 43
Validation prc-auc = 0.310490
Epoch 44
Loss = 1.7789e-01, PNorm = 81.3788, GNorm = 2.3393, lr_0 = 9.3925e-04
Validation prc-auc = 0.288724
Epoch 45
Validation prc-auc = 0.273710
Epoch 46
Validation prc-auc = 0.309345
Epoch 47
Validation prc-auc = 0.295845
Epoch 48
Validation prc-auc = 0.303109
Epoch 49
Loss = 3.0189e-01, PNorm = 81.9124, GNorm = 3.0181, lr_0 = 9.3122e-04
Validation prc-auc = 0.320837
Epoch 50
Validation prc-auc = 0.324325
Epoch 51
Validation prc-auc = 0.312761
Epoch 52
Validation prc-auc = 0.308602
Epoch 53
Validation prc-auc = 0.256017
Epoch 54
Loss = 9.6265e-02, PNorm = 82.4519, GNorm = 1.2409, lr_0 = 9.2326e-04
Validation prc-auc = 0.263593
Epoch 55
Validation prc-auc = 0.248617
Epoch 56
Validation prc-auc = 0.235027
Epoch 57
Validation prc-auc = 0.304428
Epoch 58
Validation prc-auc = 0.309252
Epoch 59
Loss = 5.3974e-02, PNorm = 82.9363, GNorm = 0.8254, lr_0 = 9.1537e-04
Validation prc-auc = 0.307894
Epoch 60
Validation prc-auc = 0.308851
Epoch 61
Validation prc-auc = 0.295125
Epoch 62
Validation prc-auc = 0.274480
Epoch 63
Validation prc-auc = 0.265507
Epoch 64
Loss = 1.0481e-01, PNorm = 83.3483, GNorm = 0.3695, lr_0 = 9.0754e-04
Validation prc-auc = 0.220221
Epoch 65
Validation prc-auc = 0.294121
Epoch 66
Validation prc-auc = 0.297513
Epoch 67
Validation prc-auc = 0.274874
Epoch 68
Validation prc-auc = 0.244755
Epoch 69
Loss = 6.7708e-02, PNorm = 83.7869, GNorm = 2.4059, lr_0 = 8.9978e-04
Validation prc-auc = 0.242615
Epoch 70
Validation prc-auc = 0.242499
Epoch 71
Validation prc-auc = 0.254930
Epoch 72
Validation prc-auc = 0.275424
Epoch 73
Validation prc-auc = 0.268193
Epoch 74
Loss = 2.9544e-02, PNorm = 84.1494, GNorm = 0.5043, lr_0 = 8.9209e-04
Validation prc-auc = 0.267116
Epoch 75
Validation prc-auc = 0.281949
Epoch 76
Validation prc-auc = 0.276047
Epoch 77
Validation prc-auc = 0.205832
Epoch 78
Validation prc-auc = 0.183349
Epoch 79
Loss = 7.6917e-02, PNorm = 84.4472, GNorm = 2.5060, lr_0 = 8.8447e-04
Validation prc-auc = 0.253312
Epoch 80
Validation prc-auc = 0.269865
Epoch 81
Validation prc-auc = 0.248978
Epoch 82
Validation prc-auc = 0.212942
Epoch 83
Validation prc-auc = 0.272852
Epoch 84
Loss = 5.6095e-02, PNorm = 84.8400, GNorm = 1.7676, lr_0 = 8.7691e-04
Validation prc-auc = 0.270776
Epoch 85
Validation prc-auc = 0.279112
Epoch 86
Validation prc-auc = 0.261560
Epoch 87
Validation prc-auc = 0.245093
Epoch 88
Validation prc-auc = 0.255377
Epoch 89
Loss = 2.8817e-02, PNorm = 85.1837, GNorm = 0.2151, lr_0 = 8.6941e-04
Validation prc-auc = 0.269885
Epoch 90
Validation prc-auc = 0.274003
Epoch 91
Validation prc-auc = 0.263056
Epoch 92
Validation prc-auc = 0.258085
Epoch 93
Validation prc-auc = 0.258102
Epoch 94
Loss = 1.4169e-02, PNorm = 85.4474, GNorm = 0.4876, lr_0 = 8.6198e-04
Validation prc-auc = 0.253735
Epoch 95
Validation prc-auc = 0.248421
Epoch 96
Validation prc-auc = 0.236368
Epoch 97
Validation prc-auc = 0.212348
Epoch 98
Validation prc-auc = 0.202274
Epoch 99
Loss = 1.1080e-02, PNorm = 85.6194, GNorm = 0.1793, lr_0 = 8.5461e-04
Validation prc-auc = 0.210346
Epoch 100
Validation prc-auc = 0.226382
Epoch 101
Validation prc-auc = 0.233868
Epoch 102
Validation prc-auc = 0.234147
Epoch 103
Validation prc-auc = 0.232578
Epoch 104
Loss = 5.8039e-03, PNorm = 85.7685, GNorm = 0.4751, lr_0 = 8.4730e-04
Validation prc-auc = 0.234665
Epoch 105
Validation prc-auc = 0.245773
Epoch 106
Validation prc-auc = 0.229765
Epoch 107
Validation prc-auc = 0.202668
Epoch 108
Validation prc-auc = 0.198487
Epoch 109
Loss = 3.3261e-03, PNorm = 85.8822, GNorm = 0.0878, lr_0 = 8.4006e-04
Validation prc-auc = 0.197556
Epoch 110
Validation prc-auc = 0.192250
Epoch 111
Validation prc-auc = 0.195907
Epoch 112
Validation prc-auc = 0.221345
Epoch 113
Validation prc-auc = 0.256502
Epoch 114
Loss = 4.8304e-03, PNorm = 85.9815, GNorm = 0.1888, lr_0 = 8.3288e-04
Validation prc-auc = 0.288530
Epoch 115
Validation prc-auc = 0.296416
Epoch 116
Validation prc-auc = 0.291004
Epoch 117
Validation prc-auc = 0.292835
Epoch 118
Validation prc-auc = 0.258954
Epoch 119
Loss = 1.6431e-03, PNorm = 86.0798, GNorm = 0.0257, lr_0 = 8.2576e-04
Validation prc-auc = 0.247764
Epoch 120
Validation prc-auc = 0.210904
Epoch 121
Validation prc-auc = 0.208080
Epoch 122
Validation prc-auc = 0.206742
Epoch 123
Validation prc-auc = 0.206940
Epoch 124
Loss = 9.5905e-04, PNorm = 86.1539, GNorm = 0.0384, lr_0 = 8.1870e-04
Validation prc-auc = 0.203504
Epoch 125
Validation prc-auc = 0.204715
Epoch 126
Validation prc-auc = 0.211334
Epoch 127
Validation prc-auc = 0.214867
Epoch 128
Validation prc-auc = 0.215314
Epoch 129
Loss = 4.8367e-04, PNorm = 86.2101, GNorm = 0.0060, lr_0 = 8.1170e-04
Validation prc-auc = 0.215197
Epoch 130
Validation prc-auc = 0.222110
Epoch 131
Validation prc-auc = 0.222112
Epoch 132
Validation prc-auc = 0.229797
Epoch 133
Validation prc-auc = 0.215422
Epoch 134
Loss = 4.2070e-03, PNorm = 86.2455, GNorm = 0.3350, lr_0 = 8.0476e-04
Validation prc-auc = 0.214897
Epoch 135
Validation prc-auc = 0.229929
Epoch 136
Validation prc-auc = 0.239647
Epoch 137
Validation prc-auc = 0.246353
Epoch 138
Validation prc-auc = 0.250734
Epoch 139
Loss = 6.0420e-04, PNorm = 86.3173, GNorm = 0.0343, lr_0 = 7.9788e-04
Validation prc-auc = 0.250647
Epoch 140
Validation prc-auc = 0.257317
Epoch 141
Validation prc-auc = 0.248711
Epoch 142
Validation prc-auc = 0.235555
Epoch 143
Validation prc-auc = 0.233822
Epoch 144
Loss = 6.8669e-04, PNorm = 86.3478, GNorm = 0.0313, lr_0 = 7.9106e-04
Validation prc-auc = 0.212841
Epoch 145
Validation prc-auc = 0.205824
Epoch 146
Validation prc-auc = 0.203542
Epoch 147
Validation prc-auc = 0.202880
Epoch 148
Validation prc-auc = 0.201224
Epoch 149
Loss = 4.4334e-04, PNorm = 86.4050, GNorm = 0.0098, lr_0 = 7.8430e-04
Validation prc-auc = 0.209669
Epoch 150
Validation prc-auc = 0.204278
Epoch 151
Validation prc-auc = 0.207228
Epoch 152
Validation prc-auc = 0.207041
Epoch 153
Validation prc-auc = 0.207041
Epoch 154
Loss = 3.5738e-04, PNorm = 86.4404, GNorm = 0.0020, lr_0 = 7.7759e-04
Validation prc-auc = 0.212598
Epoch 155
Validation prc-auc = 0.220758
Epoch 156
Validation prc-auc = 0.249183
Epoch 157
Validation prc-auc = 0.249469
Epoch 158
Validation prc-auc = 0.250900
Epoch 159
Loss = 9.2051e-04, PNorm = 86.4616, GNorm = 0.0203, lr_0 = 7.7095e-04
Validation prc-auc = 0.251085
Epoch 160
Validation prc-auc = 0.250245
Epoch 161
Validation prc-auc = 0.250634
Epoch 162
Validation prc-auc = 0.246110
Epoch 163
Validation prc-auc = 0.246093
Epoch 164
Loss = 3.2442e-04, PNorm = 86.4779, GNorm = 0.0125, lr_0 = 7.6436e-04
Validation prc-auc = 0.246118
Epoch 165
Validation prc-auc = 0.245354
Epoch 166
Validation prc-auc = 0.245917
Epoch 167
Validation prc-auc = 0.245151
Epoch 168
Validation prc-auc = 0.245481
Epoch 169
Loss = 1.9056e-04, PNorm = 86.4903, GNorm = 0.0037, lr_0 = 7.5782e-04
Validation prc-auc = 0.243320
Epoch 170
Validation prc-auc = 0.243397
Epoch 171
Validation prc-auc = 0.243613
Epoch 172
Validation prc-auc = 0.243942
Epoch 173
Validation prc-auc = 0.244307
Epoch 174
Loss = 4.2736e-04, PNorm = 86.5008, GNorm = 0.0080, lr_0 = 7.5134e-04
Validation prc-auc = 0.244400
Epoch 175
Validation prc-auc = 0.244494
Epoch 176
Validation prc-auc = 0.244454
Epoch 177
Validation prc-auc = 0.244431
Epoch 178
Validation prc-auc = 0.243393
Epoch 179
Loss = 2.6818e-04, PNorm = 86.5116, GNorm = 0.0115, lr_0 = 7.4492e-04
Validation prc-auc = 0.243267
Epoch 180
Validation prc-auc = 0.238144
Epoch 181
Validation prc-auc = 0.235473
Epoch 182
Validation prc-auc = 0.235404
Epoch 183
Validation prc-auc = 0.235608
Epoch 184
Loss = 1.8863e-04, PNorm = 86.5207, GNorm = 0.0080, lr_0 = 7.3855e-04
Validation prc-auc = 0.235624
Epoch 185
Validation prc-auc = 0.235521
Epoch 186
Validation prc-auc = 0.240956
Epoch 187
Validation prc-auc = 0.240956
Epoch 188
Validation prc-auc = 0.241229
Epoch 189
Loss = 2.0757e-04, PNorm = 86.5293, GNorm = 0.0066, lr_0 = 7.3224e-04
Validation prc-auc = 0.241229
Epoch 190
Validation prc-auc = 0.241110
Epoch 191
Validation prc-auc = 0.241110
Epoch 192
Validation prc-auc = 0.241110
Epoch 193
Validation prc-auc = 0.241021
Epoch 194
Loss = 6.9810e-05, PNorm = 86.5377, GNorm = 0.0019, lr_0 = 7.2598e-04
Validation prc-auc = 0.241021
Epoch 195
Validation prc-auc = 0.241140
Epoch 196
Validation prc-auc = 0.241040
Epoch 197
Validation prc-auc = 0.240974
Epoch 198
Validation prc-auc = 0.241132
Epoch 199
Loss = 6.9200e-05, PNorm = 86.5450, GNorm = 0.0011, lr_0 = 7.1977e-04
Validation prc-auc = 0.241248
Epoch 200
Validation prc-auc = 0.241248
Epoch 201
Validation prc-auc = 0.241436
Epoch 202
Validation prc-auc = 0.241436
Epoch 203
Validation prc-auc = 0.241436
Epoch 204
Loss = 1.2017e-04, PNorm = 86.5503, GNorm = 0.0028, lr_0 = 7.1362e-04
Validation prc-auc = 0.241436
Epoch 205
Validation prc-auc = 0.241436
Epoch 206
Validation prc-auc = 0.244757
Epoch 207
Validation prc-auc = 0.244883
Epoch 208
Validation prc-auc = 0.244796
Epoch 209
Loss = 6.0873e-05, PNorm = 86.5555, GNorm = 0.0016, lr_0 = 7.0752e-04
Validation prc-auc = 0.241373
Epoch 210
Validation prc-auc = 0.242334
Epoch 211
Validation prc-auc = 0.242443
Epoch 212
Validation prc-auc = 0.242608
Epoch 213
Validation prc-auc = 0.242856
Epoch 214
Loss = 1.3249e-04, PNorm = 86.5609, GNorm = 0.0044, lr_0 = 7.0147e-04
Validation prc-auc = 0.242639
Epoch 215
Validation prc-auc = 0.242574
Epoch 216
Validation prc-auc = 0.242574
Epoch 217
Validation prc-auc = 0.242574
Epoch 218
Validation prc-auc = 0.245897
Epoch 219
Loss = 5.6057e-05, PNorm = 86.5676, GNorm = 0.0017, lr_0 = 6.9548e-04
Validation prc-auc = 0.245913
Epoch 220
Validation prc-auc = 0.249503
Epoch 221
Validation prc-auc = 0.249583
Epoch 222
Validation prc-auc = 0.238346
Epoch 223
Validation prc-auc = 0.238017
Epoch 224
Loss = 7.1717e-05, PNorm = 86.5741, GNorm = 0.0008, lr_0 = 6.8953e-04
Validation prc-auc = 0.237636
Epoch 225
Validation prc-auc = 0.237557
Epoch 226
Validation prc-auc = 0.237557
Epoch 227
Validation prc-auc = 0.237557
Epoch 228
Validation prc-auc = 0.242161
Epoch 229
Loss = 7.9222e-05, PNorm = 86.5786, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.242221
Epoch 230
Validation prc-auc = 0.238465
Epoch 231
Validation prc-auc = 0.242052
Epoch 232
Validation prc-auc = 0.238603
Epoch 233
Validation prc-auc = 0.241255
Epoch 234
Loss = 4.5943e-05, PNorm = 86.5829, GNorm = 0.0010, lr_0 = 6.7779e-04
Validation prc-auc = 0.241878
Epoch 235
Validation prc-auc = 0.245466
Epoch 236
Validation prc-auc = 0.246428
Epoch 237
Validation prc-auc = 0.240818
Epoch 238
Validation prc-auc = 0.237349
Epoch 239
Loss = 3.8840e-05, PNorm = 86.5877, GNorm = 0.0011, lr_0 = 6.7200e-04
Validation prc-auc = 0.241673
Epoch 240
Validation prc-auc = 0.241791
Epoch 241
Validation prc-auc = 0.242033
Epoch 242
Validation prc-auc = 0.248715
Epoch 243
Validation prc-auc = 0.248478
Epoch 244
Loss = 3.0307e-05, PNorm = 86.5913, GNorm = 0.0014, lr_0 = 6.6625e-04
Validation prc-auc = 0.247657
Epoch 245
Validation prc-auc = 0.258528
Epoch 246
Validation prc-auc = 0.247285
Epoch 247
Validation prc-auc = 0.245310
Epoch 248
Validation prc-auc = 0.245393
Epoch 249
Loss = 2.7917e-05, PNorm = 86.5958, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.239596
Epoch 250
Validation prc-auc = 0.245767
Epoch 251
Validation prc-auc = 0.245592
Epoch 252
Validation prc-auc = 0.245608
Epoch 253
Validation prc-auc = 0.251031
Epoch 254
Loss = 3.1287e-05, PNorm = 86.6002, GNorm = 0.0007, lr_0 = 6.5491e-04
Validation prc-auc = 0.251048
Epoch 255
Validation prc-auc = 0.250844
Epoch 256
Validation prc-auc = 0.254301
Epoch 257
Validation prc-auc = 0.257112
Epoch 258
Validation prc-auc = 0.256776
Epoch 259
Loss = 4.4208e-05, PNorm = 86.6044, GNorm = 0.0050, lr_0 = 6.4931e-04
Validation prc-auc = 0.255416
Epoch 260
Validation prc-auc = 0.250021
Epoch 261
Validation prc-auc = 0.248634
Epoch 262
Validation prc-auc = 0.245542
Epoch 263
Validation prc-auc = 0.245542
Epoch 264
Loss = 3.8877e-05, PNorm = 86.6086, GNorm = 0.0003, lr_0 = 6.4376e-04
Validation prc-auc = 0.245999
Epoch 265
Validation prc-auc = 0.245936
Epoch 266
Validation prc-auc = 0.247678
Epoch 267
Validation prc-auc = 0.251398
Epoch 268
Validation prc-auc = 0.269290
Epoch 269
Loss = 2.9516e-05, PNorm = 86.6109, GNorm = 0.0013, lr_0 = 6.3826e-04
Validation prc-auc = 0.269625
Epoch 270
Validation prc-auc = 0.269625
Epoch 271
Validation prc-auc = 0.269516
Epoch 272
Validation prc-auc = 0.266489
Epoch 273
Validation prc-auc = 0.255238
Epoch 274
Loss = 2.3526e-05, PNorm = 86.6148, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.248307
Epoch 275
Validation prc-auc = 0.248124
Epoch 276
Validation prc-auc = 0.248124
Epoch 277
Validation prc-auc = 0.247711
Epoch 278
Validation prc-auc = 0.247905
Epoch 279
Loss = 1.7578e-05, PNorm = 86.6185, GNorm = 0.0015, lr_0 = 6.2739e-04
Validation prc-auc = 0.247873
Epoch 280
Validation prc-auc = 0.247758
Epoch 281
Validation prc-auc = 0.247821
Epoch 282
Validation prc-auc = 0.247629
Epoch 283
Validation prc-auc = 0.247742
Epoch 284
Loss = 2.6666e-05, PNorm = 86.6215, GNorm = 0.0014, lr_0 = 6.2203e-04
Validation prc-auc = 0.255118
Epoch 285
Validation prc-auc = 0.256661
Epoch 286
Validation prc-auc = 0.256779
Epoch 287
Validation prc-auc = 0.259250
Epoch 288
Validation prc-auc = 0.259301
Epoch 289
Loss = 1.7281e-05, PNorm = 86.6252, GNorm = 0.0011, lr_0 = 6.1671e-04
Validation prc-auc = 0.256731
Epoch 290
Validation prc-auc = 0.255539
Epoch 291
Validation prc-auc = 0.255066
Epoch 292
Validation prc-auc = 0.254954
Epoch 293
Validation prc-auc = 0.255534
Epoch 294
Loss = 4.2097e-05, PNorm = 86.6282, GNorm = 0.0024, lr_0 = 6.1144e-04
Validation prc-auc = 0.255490
Epoch 295
Validation prc-auc = 0.255597
Epoch 296
Validation prc-auc = 0.253429
Epoch 297
Validation prc-auc = 0.250955
Epoch 298
Validation prc-auc = 0.249664
Epoch 299
Loss = 2.6413e-05, PNorm = 86.6322, GNorm = 0.0020, lr_0 = 6.0621e-04
Validation prc-auc = 0.247300
Model 0 best validation prc-auc = 0.497692 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.401629
Ensemble test prc-auc = 0.401629
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.402364
	Seed 1 ==> test prc-auc = 0.387643
	Seed 2 ==> test prc-auc = 0.473718
	Seed 3 ==> test prc-auc = 0.528662
	Seed 4 ==> test prc-auc = 0.426883
	Seed 5 ==> test prc-auc = 0.387804
	Seed 6 ==> test prc-auc = 0.381926
	Seed 7 ==> test prc-auc = 0.512151
	Seed 8 ==> test prc-auc = 0.437878
	Seed 9 ==> test prc-auc = 0.401629
Overall test prc-auc = 0.434066 +/- 0.050750
Elapsed time = 0:09:04
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.445707
Epoch 1
Validation prc-auc = 0.458308
Epoch 2
Validation prc-auc = 0.486724
Epoch 3
Validation prc-auc = 0.446652
Epoch 4
Loss = 5.9146e-01, PNorm = 78.5516, GNorm = 6.0189, lr_0 = 6.5000e-04
Validation prc-auc = 0.423135
Epoch 5
Validation prc-auc = 0.353443
Epoch 6
Validation prc-auc = 0.357471
Epoch 7
Validation prc-auc = 0.330555
Epoch 8
Validation prc-auc = 0.281123
Epoch 9
Loss = 7.6735e-01, PNorm = 78.7998, GNorm = 20.4609, lr_0 = 9.9743e-04
Validation prc-auc = 0.403514
Epoch 10
Validation prc-auc = 0.444280
Epoch 11
Validation prc-auc = 0.486078
Epoch 12
Validation prc-auc = 0.437510
Epoch 13
Validation prc-auc = 0.431015
Epoch 14
Loss = 5.1773e-01, PNorm = 79.1107, GNorm = 1.8584, lr_0 = 9.8890e-04
Validation prc-auc = 0.330359
Epoch 15
Validation prc-auc = 0.417203
Epoch 16
Validation prc-auc = 0.406734
Epoch 17
Validation prc-auc = 0.411512
Epoch 18
Validation prc-auc = 0.442178
Epoch 19
Loss = 5.3559e-01, PNorm = 79.4173, GNorm = 5.7605, lr_0 = 9.8045e-04
Validation prc-auc = 0.451489
Epoch 20
Validation prc-auc = 0.271484
Epoch 21
Validation prc-auc = 0.438021
Epoch 22
Validation prc-auc = 0.309709
Epoch 23
Validation prc-auc = 0.164991
Epoch 24
Loss = 5.1603e-01, PNorm = 79.7061, GNorm = 3.9063, lr_0 = 9.7207e-04
Validation prc-auc = 0.326223
Epoch 25
Validation prc-auc = 0.357101
Epoch 26
Validation prc-auc = 0.349736
Epoch 27
Validation prc-auc = 0.350833
Epoch 28
Validation prc-auc = 0.347300
Epoch 29
Loss = 4.4670e-01, PNorm = 80.0649, GNorm = 5.2925, lr_0 = 9.6376e-04
Validation prc-auc = 0.339963
Epoch 30
Validation prc-auc = 0.361679
Epoch 31
Validation prc-auc = 0.340161
Epoch 32
Validation prc-auc = 0.346327
Epoch 33
Validation prc-auc = 0.357100
Epoch 34
Loss = 3.6524e-01, PNorm = 80.4179, GNorm = 1.7099, lr_0 = 9.5552e-04
Validation prc-auc = 0.340565
Epoch 35
Validation prc-auc = 0.333382
Epoch 36
Validation prc-auc = 0.303599
Epoch 37
Validation prc-auc = 0.275057
Epoch 38
Validation prc-auc = 0.351124
Epoch 39
Loss = 3.5502e-01, PNorm = 80.7368, GNorm = 3.2371, lr_0 = 9.4735e-04
Validation prc-auc = 0.341097
Epoch 40
Validation prc-auc = 0.439518
Epoch 41
Validation prc-auc = 0.368610
Epoch 42
Validation prc-auc = 0.367481
Epoch 43
Validation prc-auc = 0.225364
Epoch 44
Loss = 2.5396e-01, PNorm = 81.1163, GNorm = 0.8848, lr_0 = 9.3925e-04
Validation prc-auc = 0.315748
Epoch 45
Validation prc-auc = 0.330290
Epoch 46
Validation prc-auc = 0.349946
Epoch 47
Validation prc-auc = 0.329768
Epoch 48
Validation prc-auc = 0.347256
Epoch 49
Loss = 2.4885e-01, PNorm = 81.5223, GNorm = 4.5164, lr_0 = 9.3122e-04
Validation prc-auc = 0.308439
Epoch 50
Validation prc-auc = 0.173983
Epoch 51
Validation prc-auc = 0.259395
Epoch 52
Validation prc-auc = 0.338578
Epoch 53
Validation prc-auc = 0.360054
Epoch 54
Loss = 2.4122e-01, PNorm = 81.9848, GNorm = 1.9906, lr_0 = 9.2326e-04
Validation prc-auc = 0.347880
Epoch 55
Validation prc-auc = 0.283884
Epoch 56
Validation prc-auc = 0.305573
Epoch 57
Validation prc-auc = 0.313327
Epoch 58
Validation prc-auc = 0.337140
Epoch 59
Loss = 1.5524e-01, PNorm = 82.4705, GNorm = 3.7498, lr_0 = 9.1537e-04
Validation prc-auc = 0.340310
Epoch 60
Validation prc-auc = 0.320154
Epoch 61
Validation prc-auc = 0.354776
Epoch 62
Validation prc-auc = 0.367880
Epoch 63
Validation prc-auc = 0.358730
Epoch 64
Loss = 7.5256e-02, PNorm = 82.8959, GNorm = 0.8970, lr_0 = 9.0754e-04
Validation prc-auc = 0.372602
Epoch 65
Validation prc-auc = 0.358680
Epoch 66
Validation prc-auc = 0.293166
Epoch 67
Validation prc-auc = 0.219673
Epoch 68
Validation prc-auc = 0.296784
Epoch 69
Loss = 7.5535e-02, PNorm = 83.2407, GNorm = 1.3195, lr_0 = 8.9978e-04
Validation prc-auc = 0.295494
Epoch 70
Validation prc-auc = 0.274386
Epoch 71
Validation prc-auc = 0.297512
Epoch 72
Validation prc-auc = 0.306965
Epoch 73
Validation prc-auc = 0.283575
Epoch 74
Loss = 9.9013e-02, PNorm = 83.6169, GNorm = 2.4787, lr_0 = 8.9209e-04
Validation prc-auc = 0.239506
Epoch 75
Validation prc-auc = 0.271515
Epoch 76
Validation prc-auc = 0.263982
Epoch 77
Validation prc-auc = 0.202906
Epoch 78
Validation prc-auc = 0.298382
Epoch 79
Loss = 1.0069e-01, PNorm = 84.0248, GNorm = 1.3521, lr_0 = 8.8447e-04
Validation prc-auc = 0.336569
Epoch 80
Validation prc-auc = 0.279503
Epoch 81
Validation prc-auc = 0.299879
Epoch 82
Validation prc-auc = 0.345763
Epoch 83
Validation prc-auc = 0.325699
Epoch 84
Loss = 8.1490e-02, PNorm = 84.4339, GNorm = 2.0025, lr_0 = 8.7691e-04
Validation prc-auc = 0.271849
Epoch 85
Validation prc-auc = 0.245776
Epoch 86
Validation prc-auc = 0.263068
Epoch 87
Validation prc-auc = 0.301349
Epoch 88
Validation prc-auc = 0.292092
Epoch 89
Loss = 5.0683e-02, PNorm = 84.8066, GNorm = 0.3284, lr_0 = 8.6941e-04
Validation prc-auc = 0.284103
Epoch 90
Validation prc-auc = 0.314465
Epoch 91
Validation prc-auc = 0.327746
Epoch 92
Validation prc-auc = 0.329459
Epoch 93
Validation prc-auc = 0.303492
Epoch 94
Loss = 5.3445e-02, PNorm = 85.0653, GNorm = 0.3499, lr_0 = 8.6198e-04
Validation prc-auc = 0.277749
Epoch 95
Validation prc-auc = 0.256772
Epoch 96
Validation prc-auc = 0.278746
Epoch 97
Validation prc-auc = 0.268610
Epoch 98
Validation prc-auc = 0.264428
Epoch 99
Loss = 8.9455e-03, PNorm = 85.3073, GNorm = 0.1759, lr_0 = 8.5461e-04
Validation prc-auc = 0.270707
Epoch 100
Validation prc-auc = 0.264025
Epoch 101
Validation prc-auc = 0.272788
Epoch 102
Validation prc-auc = 0.274327
Epoch 103
Validation prc-auc = 0.266981
Epoch 104
Loss = 2.8928e-02, PNorm = 85.4931, GNorm = 0.9130, lr_0 = 8.4730e-04
Validation prc-auc = 0.258173
Epoch 105
Validation prc-auc = 0.246047
Epoch 106
Validation prc-auc = 0.245920
Epoch 107
Validation prc-auc = 0.270633
Epoch 108
Validation prc-auc = 0.285974
Epoch 109
Loss = 1.3276e-02, PNorm = 85.6793, GNorm = 0.1830, lr_0 = 8.4006e-04
Validation prc-auc = 0.284292
Epoch 110
Validation prc-auc = 0.291069
Epoch 111
Validation prc-auc = 0.279525
Epoch 112
Validation prc-auc = 0.277428
Epoch 113
Validation prc-auc = 0.279185
Epoch 114
Loss = 1.8808e-02, PNorm = 85.8312, GNorm = 0.0561, lr_0 = 8.3288e-04
Validation prc-auc = 0.276110
Epoch 115
Validation prc-auc = 0.271988
Epoch 116
Validation prc-auc = 0.271654
Epoch 117
Validation prc-auc = 0.270823
Epoch 118
Validation prc-auc = 0.270737
Epoch 119
Loss = 4.4386e-03, PNorm = 85.9360, GNorm = 0.0552, lr_0 = 8.2576e-04
Validation prc-auc = 0.265659
Epoch 120
Validation prc-auc = 0.268518
Epoch 121
Validation prc-auc = 0.263683
Epoch 122
Validation prc-auc = 0.250535
Epoch 123
Validation prc-auc = 0.229241
Epoch 124
Loss = 7.8241e-03, PNorm = 86.0179, GNorm = 0.0496, lr_0 = 8.1870e-04
Validation prc-auc = 0.205385
Epoch 125
Validation prc-auc = 0.239006
Epoch 126
Validation prc-auc = 0.258098
Epoch 127
Validation prc-auc = 0.257373
Epoch 128
Validation prc-auc = 0.263149
Epoch 129
Loss = 2.7940e-02, PNorm = 86.1505, GNorm = 0.4554, lr_0 = 8.1170e-04
Validation prc-auc = 0.271304
Epoch 130
Validation prc-auc = 0.274956
Epoch 131
Validation prc-auc = 0.271552
Epoch 132
Validation prc-auc = 0.268919
Epoch 133
Validation prc-auc = 0.270483
Epoch 134
Loss = 3.8735e-03, PNorm = 86.2603, GNorm = 0.2637, lr_0 = 8.0476e-04
Validation prc-auc = 0.276090
Epoch 135
Validation prc-auc = 0.286859
Epoch 136
Validation prc-auc = 0.281980
Epoch 137
Validation prc-auc = 0.280220
Epoch 138
Validation prc-auc = 0.269243
Epoch 139
Loss = 1.9388e-03, PNorm = 86.3415, GNorm = 0.0418, lr_0 = 7.9788e-04
Validation prc-auc = 0.268408
Epoch 140
Validation prc-auc = 0.256887
Epoch 141
Validation prc-auc = 0.260881
Epoch 142
Validation prc-auc = 0.244569
Epoch 143
Validation prc-auc = 0.241979
Epoch 144
Loss = 1.3388e-03, PNorm = 86.4374, GNorm = 0.0374, lr_0 = 7.9106e-04
Validation prc-auc = 0.231179
Epoch 145
Validation prc-auc = 0.231124
Epoch 146
Validation prc-auc = 0.244250
Epoch 147
Validation prc-auc = 0.286062
Epoch 148
Validation prc-auc = 0.272986
Epoch 149
Loss = 4.9021e-02, PNorm = 86.5866, GNorm = 2.7082, lr_0 = 7.8430e-04
Validation prc-auc = 0.232712
Epoch 150
Validation prc-auc = 0.238059
Epoch 151
Validation prc-auc = 0.248517
Epoch 152
Validation prc-auc = 0.224486
Epoch 153
Validation prc-auc = 0.212868
Epoch 154
Loss = 7.0229e-02, PNorm = 86.9283, GNorm = 0.7214, lr_0 = 7.7759e-04
Validation prc-auc = 0.217212
Epoch 155
Validation prc-auc = 0.272185
Epoch 156
Validation prc-auc = 0.293776
Epoch 157
Validation prc-auc = 0.268372
Epoch 158
Validation prc-auc = 0.227642
Epoch 159
Loss = 1.3424e-02, PNorm = 87.2404, GNorm = 0.0886, lr_0 = 7.7095e-04
Validation prc-auc = 0.232850
Epoch 160
Validation prc-auc = 0.238356
Epoch 161
Validation prc-auc = 0.235740
Epoch 162
Validation prc-auc = 0.236910
Epoch 163
Validation prc-auc = 0.235553
Epoch 164
Loss = 2.3523e-02, PNorm = 87.4460, GNorm = 0.3470, lr_0 = 7.6436e-04
Validation prc-auc = 0.231336
Epoch 165
Validation prc-auc = 0.221248
Epoch 166
Validation prc-auc = 0.219767
Epoch 167
Validation prc-auc = 0.221980
Epoch 168
Validation prc-auc = 0.219601
Epoch 169
Loss = 1.6038e-03, PNorm = 87.6006, GNorm = 0.0283, lr_0 = 7.5782e-04
Validation prc-auc = 0.229195
Epoch 170
Validation prc-auc = 0.241542
Epoch 171
Validation prc-auc = 0.245334
Epoch 172
Validation prc-auc = 0.253989
Epoch 173
Validation prc-auc = 0.266207
Epoch 174
Loss = 5.2102e-03, PNorm = 87.6887, GNorm = 0.0197, lr_0 = 7.5134e-04
Validation prc-auc = 0.264828
Epoch 175
Validation prc-auc = 0.264145
Epoch 176
Validation prc-auc = 0.271491
Epoch 177
Validation prc-auc = 0.276224
Epoch 178
Validation prc-auc = 0.275161
Epoch 179
Loss = 1.3393e-03, PNorm = 87.7670, GNorm = 0.0348, lr_0 = 7.4492e-04
Validation prc-auc = 0.274605
Epoch 180
Validation prc-auc = 0.277417
Epoch 181
Validation prc-auc = 0.294547
Epoch 182
Validation prc-auc = 0.294507
Epoch 183
Validation prc-auc = 0.301067
Epoch 184
Loss = 6.7950e-04, PNorm = 87.8157, GNorm = 0.0139, lr_0 = 7.3855e-04
Validation prc-auc = 0.301259
Epoch 185
Validation prc-auc = 0.303262
Epoch 186
Validation prc-auc = 0.286288
Epoch 187
Validation prc-auc = 0.286300
Epoch 188
Validation prc-auc = 0.295979
Epoch 189
Loss = 9.6690e-04, PNorm = 87.8530, GNorm = 0.0188, lr_0 = 7.3224e-04
Validation prc-auc = 0.301419
Epoch 190
Validation prc-auc = 0.300685
Epoch 191
Validation prc-auc = 0.294702
Epoch 192
Validation prc-auc = 0.282978
Epoch 193
Validation prc-auc = 0.270727
Epoch 194
Loss = 1.2198e-03, PNorm = 87.8885, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.266163
Epoch 195
Validation prc-auc = 0.270245
Epoch 196
Validation prc-auc = 0.270173
Epoch 197
Validation prc-auc = 0.265934
Epoch 198
Validation prc-auc = 0.270732
Epoch 199
Loss = 6.6885e-04, PNorm = 87.9168, GNorm = 0.0120, lr_0 = 7.1977e-04
Validation prc-auc = 0.271415
Epoch 200
Validation prc-auc = 0.288482
Epoch 201
Validation prc-auc = 0.284893
Epoch 202
Validation prc-auc = 0.285060
Epoch 203
Validation prc-auc = 0.307681
Epoch 204
Loss = 8.4359e-05, PNorm = 87.9388, GNorm = 0.0025, lr_0 = 7.1362e-04
Validation prc-auc = 0.314679
Epoch 205
Validation prc-auc = 0.305623
Epoch 206
Validation prc-auc = 0.310165
Epoch 207
Validation prc-auc = 0.310124
Epoch 208
Validation prc-auc = 0.301500
Epoch 209
Loss = 5.2673e-04, PNorm = 87.9587, GNorm = 0.0489, lr_0 = 7.0752e-04
Validation prc-auc = 0.300496
Epoch 210
Validation prc-auc = 0.284767
Epoch 211
Validation prc-auc = 0.279483
Epoch 212
Validation prc-auc = 0.279546
Epoch 213
Validation prc-auc = 0.279826
Epoch 214
Loss = 1.2627e-04, PNorm = 87.9753, GNorm = 0.0048, lr_0 = 7.0147e-04
Validation prc-auc = 0.284899
Epoch 215
Validation prc-auc = 0.289604
Epoch 216
Validation prc-auc = 0.300577
Epoch 217
Validation prc-auc = 0.300990
Epoch 218
Validation prc-auc = 0.301239
Epoch 219
Loss = 2.5233e-04, PNorm = 87.9907, GNorm = 0.0108, lr_0 = 6.9548e-04
Validation prc-auc = 0.301651
Epoch 220
Validation prc-auc = 0.301411
Epoch 221
Validation prc-auc = 0.301202
Epoch 222
Validation prc-auc = 0.301063
Epoch 223
Validation prc-auc = 0.300274
Epoch 224
Loss = 2.9315e-04, PNorm = 88.0053, GNorm = 0.0191, lr_0 = 6.8953e-04
Validation prc-auc = 0.295728
Epoch 225
Validation prc-auc = 0.295912
Epoch 226
Validation prc-auc = 0.291701
Epoch 227
Validation prc-auc = 0.288034
Epoch 228
Validation prc-auc = 0.291969
Epoch 229
Loss = 2.4461e-04, PNorm = 88.0127, GNorm = 0.0141, lr_0 = 6.8364e-04
Validation prc-auc = 0.296665
Epoch 230
Validation prc-auc = 0.306781
Epoch 231
Validation prc-auc = 0.311456
Epoch 232
Validation prc-auc = 0.311396
Epoch 233
Validation prc-auc = 0.311396
Epoch 234
Loss = 1.2914e-04, PNorm = 88.0232, GNorm = 0.0034, lr_0 = 6.7779e-04
Validation prc-auc = 0.303059
Epoch 235
Validation prc-auc = 0.302870
Epoch 236
Validation prc-auc = 0.302869
Epoch 237
Validation prc-auc = 0.302747
Epoch 238
Validation prc-auc = 0.302210
Epoch 239
Loss = 1.2278e-04, PNorm = 88.0353, GNorm = 0.0029, lr_0 = 6.7200e-04
Validation prc-auc = 0.302092
Epoch 240
Validation prc-auc = 0.302092
Epoch 241
Validation prc-auc = 0.297142
Epoch 242
Validation prc-auc = 0.293438
Epoch 243
Validation prc-auc = 0.293578
Epoch 244
Loss = 7.1495e-05, PNorm = 88.0434, GNorm = 0.0039, lr_0 = 6.6625e-04
Validation prc-auc = 0.293645
Epoch 245
Validation prc-auc = 0.297940
Epoch 246
Validation prc-auc = 0.298526
Epoch 247
Validation prc-auc = 0.298917
Epoch 248
Validation prc-auc = 0.302558
Epoch 249
Loss = 1.0654e-04, PNorm = 88.0486, GNorm = 0.0014, lr_0 = 6.6056e-04
Validation prc-auc = 0.313573
Epoch 250
Validation prc-auc = 0.313504
Epoch 251
Validation prc-auc = 0.313366
Epoch 252
Validation prc-auc = 0.313295
Epoch 253
Validation prc-auc = 0.302538
Epoch 254
Loss = 8.0787e-05, PNorm = 88.0568, GNorm = 0.0034, lr_0 = 6.5491e-04
Validation prc-auc = 0.298417
Epoch 255
Validation prc-auc = 0.298276
Epoch 256
Validation prc-auc = 0.297689
Epoch 257
Validation prc-auc = 0.297427
Epoch 258
Validation prc-auc = 0.297499
Epoch 259
Loss = 5.9110e-05, PNorm = 88.0638, GNorm = 0.0015, lr_0 = 6.4931e-04
Validation prc-auc = 0.298014
Epoch 260
Validation prc-auc = 0.331347
Epoch 261
Validation prc-auc = 0.331277
Epoch 262
Validation prc-auc = 0.331151
Epoch 263
Validation prc-auc = 0.331130
Epoch 264
Loss = 9.1253e-05, PNorm = 88.0694, GNorm = 0.0064, lr_0 = 6.4376e-04
Validation prc-auc = 0.335768
Epoch 265
Validation prc-auc = 0.336187
Epoch 266
Validation prc-auc = 0.335446
Epoch 267
Validation prc-auc = 0.340438
Epoch 268
Validation prc-auc = 0.340500
Epoch 269
Loss = 3.4476e-05, PNorm = 88.0747, GNorm = 0.0014, lr_0 = 6.3826e-04
Validation prc-auc = 0.340638
Epoch 270
Validation prc-auc = 0.340264
Epoch 271
Validation prc-auc = 0.336282
Epoch 272
Validation prc-auc = 0.340074
Epoch 273
Validation prc-auc = 0.340485
Epoch 274
Loss = 5.8804e-05, PNorm = 88.0807, GNorm = 0.0022, lr_0 = 6.3280e-04
Validation prc-auc = 0.346581
Epoch 275
Validation prc-auc = 0.346718
Epoch 276
Validation prc-auc = 0.356756
Epoch 277
Validation prc-auc = 0.356595
Epoch 278
Validation prc-auc = 0.356595
Epoch 279
Loss = 2.7580e-05, PNorm = 88.0861, GNorm = 0.0006, lr_0 = 6.2739e-04
Validation prc-auc = 0.356722
Epoch 280
Validation prc-auc = 0.347937
Epoch 281
Validation prc-auc = 0.347190
Epoch 282
Validation prc-auc = 0.346615
Epoch 283
Validation prc-auc = 0.346484
Epoch 284
Loss = 2.2360e-05, PNorm = 88.0910, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.346354
Epoch 285
Validation prc-auc = 0.346354
Epoch 286
Validation prc-auc = 0.340203
Epoch 287
Validation prc-auc = 0.340214
Epoch 288
Validation prc-auc = 0.335581
Epoch 289
Loss = 3.0651e-05, PNorm = 88.0939, GNorm = 0.0010, lr_0 = 6.1671e-04
Validation prc-auc = 0.335678
Epoch 290
Validation prc-auc = 0.335602
Epoch 291
Validation prc-auc = 0.335329
Epoch 292
Validation prc-auc = 0.335211
Epoch 293
Validation prc-auc = 0.335675
Epoch 294
Loss = 3.2700e-05, PNorm = 88.0977, GNorm = 0.0052, lr_0 = 6.1144e-04
Validation prc-auc = 0.335876
Epoch 295
Validation prc-auc = 0.335479
Epoch 296
Validation prc-auc = 0.335553
Epoch 297
Validation prc-auc = 0.335358
Epoch 298
Validation prc-auc = 0.335749
Epoch 299
Loss = 4.0968e-05, PNorm = 88.1015, GNorm = 0.0011, lr_0 = 6.0621e-04
Validation prc-auc = 0.347635
Model 0 best validation prc-auc = 0.486724 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.402364
Ensemble test prc-auc = 0.402364
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.454787
Epoch 1
Validation prc-auc = 0.455183
Epoch 2
Validation prc-auc = 0.457246
Epoch 3
Validation prc-auc = 0.460642
Epoch 4
Loss = 7.0194e-01, PNorm = 78.5504, GNorm = 3.9290, lr_0 = 6.5000e-04
Validation prc-auc = 0.442238
Epoch 5
Validation prc-auc = 0.453212
Epoch 6
Validation prc-auc = 0.457813
Epoch 7
Validation prc-auc = 0.462085
Epoch 8
Validation prc-auc = 0.472303
Epoch 9
Loss = 6.0939e-01, PNorm = 78.7958, GNorm = 9.0791, lr_0 = 9.9743e-04
Validation prc-auc = 0.462185
Epoch 10
Validation prc-auc = 0.334396
Epoch 11
Validation prc-auc = 0.306692
Epoch 12
Validation prc-auc = 0.346881
Epoch 13
Validation prc-auc = 0.310795
Epoch 14
Loss = 5.8294e-01, PNorm = 79.0813, GNorm = 2.1918, lr_0 = 9.8890e-04
Validation prc-auc = 0.269379
Epoch 15
Validation prc-auc = 0.337045
Epoch 16
Validation prc-auc = 0.434443
Epoch 17
Validation prc-auc = 0.441915
Epoch 18
Validation prc-auc = 0.418510
Epoch 19
Loss = 4.2754e-01, PNorm = 79.3729, GNorm = 1.9741, lr_0 = 9.8045e-04
Validation prc-auc = 0.392791
Epoch 20
Validation prc-auc = 0.420089
Epoch 21
Validation prc-auc = 0.403046
Epoch 22
Validation prc-auc = 0.367614
Epoch 23
Validation prc-auc = 0.337152
Epoch 24
Loss = 4.9482e-01, PNorm = 79.6858, GNorm = 6.3413, lr_0 = 9.7207e-04
Validation prc-auc = 0.267790
Epoch 25
Validation prc-auc = 0.271424
Epoch 26
Validation prc-auc = 0.276685
Epoch 27
Validation prc-auc = 0.296748
Epoch 28
Validation prc-auc = 0.338195
Epoch 29
Loss = 3.4506e-01, PNorm = 80.0312, GNorm = 0.9223, lr_0 = 9.6376e-04
Validation prc-auc = 0.416699
Epoch 30
Validation prc-auc = 0.457941
Epoch 31
Validation prc-auc = 0.450469
Epoch 32
Validation prc-auc = 0.270926
Epoch 33
Validation prc-auc = 0.334869
Epoch 34
Loss = 2.5451e-01, PNorm = 80.3966, GNorm = 2.3773, lr_0 = 9.5552e-04
Validation prc-auc = 0.389059
Epoch 35
Validation prc-auc = 0.405813
Epoch 36
Validation prc-auc = 0.377166
Epoch 37
Validation prc-auc = 0.340674
Epoch 38
Validation prc-auc = 0.234162
Epoch 39
Loss = 2.9442e-01, PNorm = 80.7763, GNorm = 2.9394, lr_0 = 9.4735e-04
Validation prc-auc = 0.199282
Epoch 40
Validation prc-auc = 0.199382
Epoch 41
Validation prc-auc = 0.341877
Epoch 42
Validation prc-auc = 0.327700
Epoch 43
Validation prc-auc = 0.348898
Epoch 44
Loss = 1.9134e-01, PNorm = 81.2134, GNorm = 4.3996, lr_0 = 9.3925e-04
Validation prc-auc = 0.336673
Epoch 45
Validation prc-auc = 0.316691
Epoch 46
Validation prc-auc = 0.336910
Epoch 47
Validation prc-auc = 0.262107
Epoch 48
Validation prc-auc = 0.213369
Epoch 49
Loss = 1.0014e-01, PNorm = 81.6307, GNorm = 1.4613, lr_0 = 9.3122e-04
Validation prc-auc = 0.276573
Epoch 50
Validation prc-auc = 0.293355
Epoch 51
Validation prc-auc = 0.294258
Epoch 52
Validation prc-auc = 0.324556
Epoch 53
Validation prc-auc = 0.287274
Epoch 54
Loss = 3.0780e-01, PNorm = 82.0069, GNorm = 3.1248, lr_0 = 9.2326e-04
Validation prc-auc = 0.275376
Epoch 55
Validation prc-auc = 0.299874
Epoch 56
Validation prc-auc = 0.317530
Epoch 57
Validation prc-auc = 0.309956
Epoch 58
Validation prc-auc = 0.262078
Epoch 59
Loss = 1.6468e-01, PNorm = 82.3623, GNorm = 0.7541, lr_0 = 9.1537e-04
Validation prc-auc = 0.258190
Epoch 60
Validation prc-auc = 0.233495
Epoch 61
Validation prc-auc = 0.216412
Epoch 62
Validation prc-auc = 0.240594
Epoch 63
Validation prc-auc = 0.211781
Epoch 64
Loss = 1.1436e-01, PNorm = 82.7665, GNorm = 2.5405, lr_0 = 9.0754e-04
Validation prc-auc = 0.201018
Epoch 65
Validation prc-auc = 0.215336
Epoch 66
Validation prc-auc = 0.193294
Epoch 67
Validation prc-auc = 0.243513
Epoch 68
Validation prc-auc = 0.351066
Epoch 69
Loss = 1.5013e-01, PNorm = 83.1409, GNorm = 0.7780, lr_0 = 8.9978e-04
Validation prc-auc = 0.197157
Epoch 70
Validation prc-auc = 0.267839
Epoch 71
Validation prc-auc = 0.256646
Epoch 72
Validation prc-auc = 0.227305
Epoch 73
Validation prc-auc = 0.221294
Epoch 74
Loss = 8.8397e-02, PNorm = 83.4788, GNorm = 0.3822, lr_0 = 8.9209e-04
Validation prc-auc = 0.243400
Epoch 75
Validation prc-auc = 0.257978
Epoch 76
Validation prc-auc = 0.249578
Epoch 77
Validation prc-auc = 0.253461
Epoch 78
Validation prc-auc = 0.267424
Epoch 79
Loss = 5.1197e-02, PNorm = 83.7954, GNorm = 0.5032, lr_0 = 8.8447e-04
Validation prc-auc = 0.286060
Epoch 80
Validation prc-auc = 0.258376
Epoch 81
Validation prc-auc = 0.256599
Epoch 82
Validation prc-auc = 0.249125
Epoch 83
Validation prc-auc = 0.232321
Epoch 84
Loss = 2.3021e-02, PNorm = 84.0485, GNorm = 0.8325, lr_0 = 8.7691e-04
Validation prc-auc = 0.240687
Epoch 85
Validation prc-auc = 0.241343
Epoch 86
Validation prc-auc = 0.208891
Epoch 87
Validation prc-auc = 0.189567
Epoch 88
Validation prc-auc = 0.199458
Epoch 89
Loss = 3.3585e-02, PNorm = 84.2406, GNorm = 1.3216, lr_0 = 8.6941e-04
Validation prc-auc = 0.227136
Epoch 90
Validation prc-auc = 0.251822
Epoch 91
Validation prc-auc = 0.242586
Epoch 92
Validation prc-auc = 0.225761
Epoch 93
Validation prc-auc = 0.209611
Epoch 94
Loss = 1.7099e-02, PNorm = 84.4713, GNorm = 0.4374, lr_0 = 8.6198e-04
Validation prc-auc = 0.200627
Epoch 95
Validation prc-auc = 0.239579
Epoch 96
Validation prc-auc = 0.252516
Epoch 97
Validation prc-auc = 0.266980
Epoch 98
Validation prc-auc = 0.286916
Epoch 99
Loss = 7.2256e-03, PNorm = 84.6944, GNorm = 0.2185, lr_0 = 8.5461e-04
Validation prc-auc = 0.287583
Epoch 100
Validation prc-auc = 0.251571
Epoch 101
Validation prc-auc = 0.241074
Epoch 102
Validation prc-auc = 0.205227
Epoch 103
Validation prc-auc = 0.171208
Epoch 104
Loss = 5.0183e-02, PNorm = 84.8603, GNorm = 3.1848, lr_0 = 8.4730e-04
Validation prc-auc = 0.210539
Epoch 105
Validation prc-auc = 0.259053
Epoch 106
Validation prc-auc = 0.281646
Epoch 107
Validation prc-auc = 0.276828
Epoch 108
Validation prc-auc = 0.281906
Epoch 109
Loss = 1.3714e-02, PNorm = 85.0971, GNorm = 0.2898, lr_0 = 8.4006e-04
Validation prc-auc = 0.301848
Epoch 110
Validation prc-auc = 0.310100
Epoch 111
Validation prc-auc = 0.299987
Epoch 112
Validation prc-auc = 0.247440
Epoch 113
Validation prc-auc = 0.216580
Epoch 114
Loss = 1.5790e-02, PNorm = 85.3088, GNorm = 0.0343, lr_0 = 8.3288e-04
Validation prc-auc = 0.197170
Epoch 115
Validation prc-auc = 0.193215
Epoch 116
Validation prc-auc = 0.207562
Epoch 117
Validation prc-auc = 0.219012
Epoch 118
Validation prc-auc = 0.227806
Epoch 119
Loss = 7.5661e-03, PNorm = 85.5044, GNorm = 0.2694, lr_0 = 8.2576e-04
Validation prc-auc = 0.224288
Epoch 120
Validation prc-auc = 0.251187
Epoch 121
Validation prc-auc = 0.226028
Epoch 122
Validation prc-auc = 0.220938
Epoch 123
Validation prc-auc = 0.228040
Epoch 124
Loss = 5.3320e-02, PNorm = 85.6556, GNorm = 1.0852, lr_0 = 8.1870e-04
Validation prc-auc = 0.196481
Epoch 125
Validation prc-auc = 0.205610
Epoch 126
Validation prc-auc = 0.236184
Epoch 127
Validation prc-auc = 0.241849
Epoch 128
Validation prc-auc = 0.259201
Epoch 129
Loss = 3.4204e-02, PNorm = 85.8841, GNorm = 1.9203, lr_0 = 8.1170e-04
Validation prc-auc = 0.260198
Epoch 130
Validation prc-auc = 0.256769
Epoch 131
Validation prc-auc = 0.205087
Epoch 132
Validation prc-auc = 0.194703
Epoch 133
Validation prc-auc = 0.222715
Epoch 134
Loss = 1.1991e-02, PNorm = 86.0542, GNorm = 0.4515, lr_0 = 8.0476e-04
Validation prc-auc = 0.240268
Epoch 135
Validation prc-auc = 0.257440
Epoch 136
Validation prc-auc = 0.261329
Epoch 137
Validation prc-auc = 0.262965
Epoch 138
Validation prc-auc = 0.301042
Epoch 139
Loss = 3.7627e-03, PNorm = 86.2992, GNorm = 0.1044, lr_0 = 7.9788e-04
Validation prc-auc = 0.311650
Epoch 140
Validation prc-auc = 0.320001
Epoch 141
Validation prc-auc = 0.306695
Epoch 142
Validation prc-auc = 0.270824
Epoch 143
Validation prc-auc = 0.253500
Epoch 144
Loss = 7.5574e-03, PNorm = 86.4422, GNorm = 0.2978, lr_0 = 7.9106e-04
Validation prc-auc = 0.244863
Epoch 145
Validation prc-auc = 0.241338
Epoch 146
Validation prc-auc = 0.240851
Epoch 147
Validation prc-auc = 0.241389
Epoch 148
Validation prc-auc = 0.257856
Epoch 149
Loss = 7.9625e-03, PNorm = 86.6006, GNorm = 0.6261, lr_0 = 7.8430e-04
Validation prc-auc = 0.252878
Epoch 150
Validation prc-auc = 0.247248
Epoch 151
Validation prc-auc = 0.250543
Epoch 152
Validation prc-auc = 0.210007
Epoch 153
Validation prc-auc = 0.196377
Epoch 154
Loss = 4.1447e-03, PNorm = 86.7082, GNorm = 0.1345, lr_0 = 7.7759e-04
Validation prc-auc = 0.181752
Epoch 155
Validation prc-auc = 0.175607
Epoch 156
Validation prc-auc = 0.172722
Epoch 157
Validation prc-auc = 0.192026
Epoch 158
Validation prc-auc = 0.201716
Epoch 159
Loss = 1.8512e-03, PNorm = 86.7960, GNorm = 0.0608, lr_0 = 7.7095e-04
Validation prc-auc = 0.215760
Epoch 160
Validation prc-auc = 0.221667
Epoch 161
Validation prc-auc = 0.220881
Epoch 162
Validation prc-auc = 0.226931
Epoch 163
Validation prc-auc = 0.226766
Epoch 164
Loss = 1.4032e-03, PNorm = 86.8615, GNorm = 0.0165, lr_0 = 7.6436e-04
Validation prc-auc = 0.230959
Epoch 165
Validation prc-auc = 0.230962
Epoch 166
Validation prc-auc = 0.231031
Epoch 167
Validation prc-auc = 0.230962
Epoch 168
Validation prc-auc = 0.231195
Epoch 169
Loss = 8.6651e-04, PNorm = 86.8980, GNorm = 0.0151, lr_0 = 7.5782e-04
Validation prc-auc = 0.232348
Epoch 170
Validation prc-auc = 0.233524
Epoch 171
Validation prc-auc = 0.233524
Epoch 172
Validation prc-auc = 0.231752
Epoch 173
Validation prc-auc = 0.231915
Epoch 174
Loss = 3.3568e-04, PNorm = 86.9257, GNorm = 0.0100, lr_0 = 7.5134e-04
Validation prc-auc = 0.230779
Epoch 175
Validation prc-auc = 0.230578
Epoch 176
Validation prc-auc = 0.229896
Epoch 177
Validation prc-auc = 0.227245
Epoch 178
Validation prc-auc = 0.231205
Epoch 179
Loss = 1.4119e-03, PNorm = 86.9472, GNorm = 0.1662, lr_0 = 7.4492e-04
Validation prc-auc = 0.233057
Epoch 180
Validation prc-auc = 0.229963
Epoch 181
Validation prc-auc = 0.230312
Epoch 182
Validation prc-auc = 0.232165
Epoch 183
Validation prc-auc = 0.232298
Epoch 184
Loss = 4.3759e-04, PNorm = 86.9597, GNorm = 0.0093, lr_0 = 7.3855e-04
Validation prc-auc = 0.231458
Epoch 185
Validation prc-auc = 0.231291
Epoch 186
Validation prc-auc = 0.231148
Epoch 187
Validation prc-auc = 0.231077
Epoch 188
Validation prc-auc = 0.230981
Epoch 189
Loss = 3.5932e-04, PNorm = 86.9807, GNorm = 0.0078, lr_0 = 7.3224e-04
Validation prc-auc = 0.230504
Epoch 190
Validation prc-auc = 0.226111
Epoch 191
Validation prc-auc = 0.226568
Epoch 192
Validation prc-auc = 0.225758
Epoch 193
Validation prc-auc = 0.225825
Epoch 194
Loss = 3.8743e-04, PNorm = 86.9999, GNorm = 0.0089, lr_0 = 7.2598e-04
Validation prc-auc = 0.225745
Epoch 195
Validation prc-auc = 0.228954
Epoch 196
Validation prc-auc = 0.228890
Epoch 197
Validation prc-auc = 0.230440
Epoch 198
Validation prc-auc = 0.230437
Epoch 199
Loss = 1.7005e-04, PNorm = 87.0120, GNorm = 0.0049, lr_0 = 7.1977e-04
Validation prc-auc = 0.231265
Epoch 200
Validation prc-auc = 0.231446
Epoch 201
Validation prc-auc = 0.231379
Epoch 202
Validation prc-auc = 0.235159
Epoch 203
Validation prc-auc = 0.235159
Epoch 204
Loss = 1.4709e-04, PNorm = 87.0239, GNorm = 0.0044, lr_0 = 7.1362e-04
Validation prc-auc = 0.235159
Epoch 205
Validation prc-auc = 0.238190
Epoch 206
Validation prc-auc = 0.238191
Epoch 207
Validation prc-auc = 0.235292
Epoch 208
Validation prc-auc = 0.235292
Epoch 209
Loss = 2.4106e-04, PNorm = 87.0373, GNorm = 0.0197, lr_0 = 7.0752e-04
Validation prc-auc = 0.235292
Epoch 210
Validation prc-auc = 0.235228
Epoch 211
Validation prc-auc = 0.235247
Epoch 212
Validation prc-auc = 0.243912
Epoch 213
Validation prc-auc = 0.245091
Epoch 214
Loss = 8.8845e-05, PNorm = 87.0463, GNorm = 0.0031, lr_0 = 7.0147e-04
Validation prc-auc = 0.245023
Epoch 215
Validation prc-auc = 0.245093
Epoch 216
Validation prc-auc = 0.245093
Epoch 217
Validation prc-auc = 0.247590
Epoch 218
Validation prc-auc = 0.247634
Epoch 219
Loss = 1.6314e-04, PNorm = 87.0580, GNorm = 0.0022, lr_0 = 6.9548e-04
Validation prc-auc = 0.246717
Epoch 220
Validation prc-auc = 0.246717
Epoch 221
Validation prc-auc = 0.246717
Epoch 222
Validation prc-auc = 0.251977
Epoch 223
Validation prc-auc = 0.251886
Epoch 224
Loss = 5.7891e-04, PNorm = 87.0689, GNorm = 0.0141, lr_0 = 6.8953e-04
Validation prc-auc = 0.252070
Epoch 225
Validation prc-auc = 0.252165
Epoch 226
Validation prc-auc = 0.252196
Epoch 227
Validation prc-auc = 0.252196
Epoch 228
Validation prc-auc = 0.252120
Epoch 229
Loss = 9.1246e-05, PNorm = 87.0767, GNorm = 0.0028, lr_0 = 6.8364e-04
Validation prc-auc = 0.252186
Epoch 230
Validation prc-auc = 0.252196
Epoch 231
Validation prc-auc = 0.252196
Epoch 232
Validation prc-auc = 0.252196
Epoch 233
Validation prc-auc = 0.259637
Epoch 234
Loss = 1.0015e-04, PNorm = 87.0887, GNorm = 0.0019, lr_0 = 6.7779e-04
Validation prc-auc = 0.259478
Epoch 235
Validation prc-auc = 0.259461
Epoch 236
Validation prc-auc = 0.259640
Epoch 237
Validation prc-auc = 0.259782
Epoch 238
Validation prc-auc = 0.260681
Epoch 239
Loss = 6.6957e-05, PNorm = 87.0977, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.260835
Epoch 240
Validation prc-auc = 0.260910
Epoch 241
Validation prc-auc = 0.264819
Epoch 242
Validation prc-auc = 0.265838
Epoch 243
Validation prc-auc = 0.265932
Epoch 244
Loss = 5.9781e-05, PNorm = 87.1072, GNorm = 0.0013, lr_0 = 6.6625e-04
Validation prc-auc = 0.265996
Epoch 245
Validation prc-auc = 0.265902
Epoch 246
Validation prc-auc = 0.264921
Epoch 247
Validation prc-auc = 0.264822
Epoch 248
Validation prc-auc = 0.264643
Epoch 249
Loss = 4.1430e-05, PNorm = 87.1184, GNorm = 0.0010, lr_0 = 6.6056e-04
Validation prc-auc = 0.263816
Epoch 250
Validation prc-auc = 0.264666
Epoch 251
Validation prc-auc = 0.263755
Epoch 252
Validation prc-auc = 0.263099
Epoch 253
Validation prc-auc = 0.263928
Epoch 254
Loss = 4.1046e-05, PNorm = 87.1295, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.264008
Epoch 255
Validation prc-auc = 0.265221
Epoch 256
Validation prc-auc = 0.266494
Epoch 257
Validation prc-auc = 0.266876
Epoch 258
Validation prc-auc = 0.266876
Epoch 259
Loss = 4.6054e-05, PNorm = 87.1382, GNorm = 0.0006, lr_0 = 6.4931e-04
Validation prc-auc = 0.265720
Epoch 260
Validation prc-auc = 0.264760
Epoch 261
Validation prc-auc = 0.265828
Epoch 262
Validation prc-auc = 0.267399
Epoch 263
Validation prc-auc = 0.268491
Epoch 264
Loss = 2.0171e-05, PNorm = 87.1470, GNorm = 0.0006, lr_0 = 6.4376e-04
Validation prc-auc = 0.268647
Epoch 265
Validation prc-auc = 0.268741
Epoch 266
Validation prc-auc = 0.267219
Epoch 267
Validation prc-auc = 0.268426
Epoch 268
Validation prc-auc = 0.263798
Epoch 269
Loss = 1.1883e-05, PNorm = 87.1543, GNorm = 0.0009, lr_0 = 6.3826e-04
Validation prc-auc = 0.262209
Epoch 270
Validation prc-auc = 0.263728
Epoch 271
Validation prc-auc = 0.267963
Epoch 272
Validation prc-auc = 0.276863
Epoch 273
Validation prc-auc = 0.281979
Epoch 274
Loss = 1.8071e-05, PNorm = 87.1597, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.282208
Epoch 275
Validation prc-auc = 0.273036
Epoch 276
Validation prc-auc = 0.271376
Epoch 277
Validation prc-auc = 0.270654
Epoch 278
Validation prc-auc = 0.267815
Epoch 279
Loss = 1.2040e-05, PNorm = 87.1653, GNorm = 0.0022, lr_0 = 6.2739e-04
Validation prc-auc = 0.254367
Epoch 280
Validation prc-auc = 0.248043
Epoch 281
Validation prc-auc = 0.254376
Epoch 282
Validation prc-auc = 0.255854
Epoch 283
Validation prc-auc = 0.257660
Epoch 284
Loss = 1.3937e-05, PNorm = 87.1688, GNorm = 0.0005, lr_0 = 6.2203e-04
Validation prc-auc = 0.261545
Epoch 285
Validation prc-auc = 0.264887
Epoch 286
Validation prc-auc = 0.264821
Epoch 287
Validation prc-auc = 0.264757
Epoch 288
Validation prc-auc = 0.262994
Epoch 289
Loss = 7.4108e-06, PNorm = 87.1729, GNorm = 0.0005, lr_0 = 6.1671e-04
Validation prc-auc = 0.263275
Epoch 290
Validation prc-auc = 0.262037
Epoch 291
Validation prc-auc = 0.262037
Epoch 292
Validation prc-auc = 0.261647
Epoch 293
Validation prc-auc = 0.261562
Epoch 294
Loss = 8.6336e-06, PNorm = 87.1768, GNorm = 0.0006, lr_0 = 6.1144e-04
Validation prc-auc = 0.257813
Epoch 295
Validation prc-auc = 0.280159
Epoch 296
Validation prc-auc = 0.282587
Epoch 297
Validation prc-auc = 0.285804
Epoch 298
Validation prc-auc = 0.282537
Epoch 299
Loss = 2.0865e-05, PNorm = 87.1799, GNorm = 0.0019, lr_0 = 6.0621e-04
Validation prc-auc = 0.281201
Model 0 best validation prc-auc = 0.472303 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387643
Ensemble test prc-auc = 0.387643
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.434076
Epoch 1
Validation prc-auc = 0.457420
Epoch 2
Validation prc-auc = 0.458604
Epoch 3
Validation prc-auc = 0.471728
Epoch 4
Loss = 6.6940e-01, PNorm = 78.5476, GNorm = 6.7897, lr_0 = 6.5000e-04
Validation prc-auc = 0.459812
Epoch 5
Validation prc-auc = 0.441032
Epoch 6
Validation prc-auc = 0.481371
Epoch 7
Validation prc-auc = 0.465603
Epoch 8
Validation prc-auc = 0.490614
Epoch 9
Loss = 7.5357e-01, PNorm = 78.7818, GNorm = 4.4542, lr_0 = 9.9743e-04
Validation prc-auc = 0.478565
Epoch 10
Validation prc-auc = 0.321278
Epoch 11
Validation prc-auc = 0.329693
Epoch 12
Validation prc-auc = 0.402455
Epoch 13
Validation prc-auc = 0.457976
Epoch 14
Loss = 4.9512e-01, PNorm = 79.0603, GNorm = 4.6236, lr_0 = 9.8890e-04
Validation prc-auc = 0.465991
Epoch 15
Validation prc-auc = 0.474146
Epoch 16
Validation prc-auc = 0.494079
Epoch 17
Validation prc-auc = 0.481656
Epoch 18
Validation prc-auc = 0.491480
Epoch 19
Loss = 6.0016e-01, PNorm = 79.3179, GNorm = 11.3375, lr_0 = 9.8045e-04
Validation prc-auc = 0.356324
Epoch 20
Validation prc-auc = 0.319028
Epoch 21
Validation prc-auc = 0.396666
Epoch 22
Validation prc-auc = 0.327262
Epoch 23
Validation prc-auc = 0.249020
Epoch 24
Loss = 5.2568e-01, PNorm = 79.5741, GNorm = 3.9257, lr_0 = 9.7207e-04
Validation prc-auc = 0.344926
Epoch 25
Validation prc-auc = 0.463539
Epoch 26
Validation prc-auc = 0.462075
Epoch 27
Validation prc-auc = 0.463508
Epoch 28
Validation prc-auc = 0.462585
Epoch 29
Loss = 4.8583e-01, PNorm = 79.8787, GNorm = 1.1889, lr_0 = 9.6376e-04
Validation prc-auc = 0.441641
Epoch 30
Validation prc-auc = 0.350212
Epoch 31
Validation prc-auc = 0.307020
Epoch 32
Validation prc-auc = 0.295028
Epoch 33
Validation prc-auc = 0.308890
Epoch 34
Loss = 3.5980e-01, PNorm = 80.1620, GNorm = 1.2794, lr_0 = 9.5552e-04
Validation prc-auc = 0.347034
Epoch 35
Validation prc-auc = 0.345844
Epoch 36
Validation prc-auc = 0.329048
Epoch 37
Validation prc-auc = 0.338035
Epoch 38
Validation prc-auc = 0.385873
Epoch 39
Loss = 3.6370e-01, PNorm = 80.4752, GNorm = 3.3556, lr_0 = 9.4735e-04
Validation prc-auc = 0.354778
Epoch 40
Validation prc-auc = 0.310202
Epoch 41
Validation prc-auc = 0.297669
Epoch 42
Validation prc-auc = 0.324231
Epoch 43
Validation prc-auc = 0.296734
Epoch 44
Loss = 3.6850e-01, PNorm = 80.7925, GNorm = 5.2399, lr_0 = 9.3925e-04
Validation prc-auc = 0.279638
Epoch 45
Validation prc-auc = 0.334626
Epoch 46
Validation prc-auc = 0.341671
Epoch 47
Validation prc-auc = 0.358256
Epoch 48
Validation prc-auc = 0.330206
Epoch 49
Loss = 2.1353e-01, PNorm = 81.1628, GNorm = 2.3248, lr_0 = 9.3122e-04
Validation prc-auc = 0.352046
Epoch 50
Validation prc-auc = 0.303014
Epoch 51
Validation prc-auc = 0.310926
Epoch 52
Validation prc-auc = 0.307721
Epoch 53
Validation prc-auc = 0.320798
Epoch 54
Loss = 2.5593e-01, PNorm = 81.5512, GNorm = 5.2760, lr_0 = 9.2326e-04
Validation prc-auc = 0.343211
Epoch 55
Validation prc-auc = 0.337257
Epoch 56
Validation prc-auc = 0.359772
Epoch 57
Validation prc-auc = 0.377172
Epoch 58
Validation prc-auc = 0.342004
Epoch 59
Loss = 1.8163e-01, PNorm = 81.9307, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation prc-auc = 0.298545
Epoch 60
Validation prc-auc = 0.280353
Epoch 61
Validation prc-auc = 0.294281
Epoch 62
Validation prc-auc = 0.265543
Epoch 63
Validation prc-auc = 0.258436
Epoch 64
Loss = 1.3860e-01, PNorm = 82.3406, GNorm = 1.8254, lr_0 = 9.0754e-04
Validation prc-auc = 0.274408
Epoch 65
Validation prc-auc = 0.191392
Epoch 66
Validation prc-auc = 0.298554
Epoch 67
Validation prc-auc = 0.304117
Epoch 68
Validation prc-auc = 0.362464
Epoch 69
Loss = 6.8738e-02, PNorm = 82.7452, GNorm = 1.6550, lr_0 = 8.9978e-04
Validation prc-auc = 0.373048
Epoch 70
Validation prc-auc = 0.370687
Epoch 71
Validation prc-auc = 0.368884
Epoch 72
Validation prc-auc = 0.334850
Epoch 73
Validation prc-auc = 0.307704
Epoch 74
Loss = 6.8877e-02, PNorm = 83.0944, GNorm = 0.3143, lr_0 = 8.9209e-04
Validation prc-auc = 0.269460
Epoch 75
Validation prc-auc = 0.229206
Epoch 76
Validation prc-auc = 0.269542
Epoch 77
Validation prc-auc = 0.207595
Epoch 78
Validation prc-auc = 0.290756
Epoch 79
Loss = 8.6926e-02, PNorm = 83.4015, GNorm = 0.5161, lr_0 = 8.8447e-04
Validation prc-auc = 0.283063
Epoch 80
Validation prc-auc = 0.283195
Epoch 81
Validation prc-auc = 0.328266
Epoch 82
Validation prc-auc = 0.342782
Epoch 83
Validation prc-auc = 0.306091
Epoch 84
Loss = 2.3302e-01, PNorm = 83.7335, GNorm = 1.7786, lr_0 = 8.7691e-04
Validation prc-auc = 0.279411
Epoch 85
Validation prc-auc = 0.281117
Epoch 86
Validation prc-auc = 0.246712
Epoch 87
Validation prc-auc = 0.224165
Epoch 88
Validation prc-auc = 0.287092
Epoch 89
Loss = 4.5509e-02, PNorm = 84.1042, GNorm = 0.6834, lr_0 = 8.6941e-04
Validation prc-auc = 0.308344
Epoch 90
Validation prc-auc = 0.328625
Epoch 91
Validation prc-auc = 0.309906
Epoch 92
Validation prc-auc = 0.261949
Epoch 93
Validation prc-auc = 0.238334
Epoch 94
Loss = 2.9086e-02, PNorm = 84.3840, GNorm = 0.7699, lr_0 = 8.6198e-04
Validation prc-auc = 0.233986
Epoch 95
Validation prc-auc = 0.242076
Epoch 96
Validation prc-auc = 0.234171
Epoch 97
Validation prc-auc = 0.259769
Epoch 98
Validation prc-auc = 0.260372
Epoch 99
Loss = 2.7894e-02, PNorm = 84.6258, GNorm = 0.5002, lr_0 = 8.5461e-04
Validation prc-auc = 0.241337
Epoch 100
Validation prc-auc = 0.230159
Epoch 101
Validation prc-auc = 0.240429
Epoch 102
Validation prc-auc = 0.245848
Epoch 103
Validation prc-auc = 0.256310
Epoch 104
Loss = 4.0704e-02, PNorm = 84.8366, GNorm = 1.1440, lr_0 = 8.4730e-04
Validation prc-auc = 0.229493
Epoch 105
Validation prc-auc = 0.226121
Epoch 106
Validation prc-auc = 0.222959
Epoch 107
Validation prc-auc = 0.226675
Epoch 108
Validation prc-auc = 0.242013
Epoch 109
Loss = 1.4584e-02, PNorm = 84.9962, GNorm = 0.6123, lr_0 = 8.4006e-04
Validation prc-auc = 0.256096
Epoch 110
Validation prc-auc = 0.262724
Epoch 111
Validation prc-auc = 0.237542
Epoch 112
Validation prc-auc = 0.212855
Epoch 113
Validation prc-auc = 0.222727
Epoch 114
Loss = 2.4388e-02, PNorm = 85.1564, GNorm = 0.9167, lr_0 = 8.3288e-04
Validation prc-auc = 0.242274
Epoch 115
Validation prc-auc = 0.279204
Epoch 116
Validation prc-auc = 0.278679
Epoch 117
Validation prc-auc = 0.253609
Epoch 118
Validation prc-auc = 0.214742
Epoch 119
Loss = 3.5193e-02, PNorm = 85.3554, GNorm = 0.8935, lr_0 = 8.2576e-04
Validation prc-auc = 0.189706
Epoch 120
Validation prc-auc = 0.186311
Epoch 121
Validation prc-auc = 0.245165
Epoch 122
Validation prc-auc = 0.237705
Epoch 123
Validation prc-auc = 0.240659
Epoch 124
Loss = 2.4166e-02, PNorm = 85.5488, GNorm = 0.2369, lr_0 = 8.1870e-04
Validation prc-auc = 0.250475
Epoch 125
Validation prc-auc = 0.266610
Epoch 126
Validation prc-auc = 0.266784
Epoch 127
Validation prc-auc = 0.270365
Epoch 128
Validation prc-auc = 0.245779
Epoch 129
Loss = 7.9705e-03, PNorm = 85.7155, GNorm = 0.1555, lr_0 = 8.1170e-04
Validation prc-auc = 0.231842
Epoch 130
Validation prc-auc = 0.214591
Epoch 131
Validation prc-auc = 0.212922
Epoch 132
Validation prc-auc = 0.210431
Epoch 133
Validation prc-auc = 0.209643
Epoch 134
Loss = 3.5030e-03, PNorm = 85.8338, GNorm = 0.2374, lr_0 = 8.0476e-04
Validation prc-auc = 0.207306
Epoch 135
Validation prc-auc = 0.200479
Epoch 136
Validation prc-auc = 0.202470
Epoch 137
Validation prc-auc = 0.205772
Epoch 138
Validation prc-auc = 0.209165
Epoch 139
Loss = 3.8180e-03, PNorm = 85.9040, GNorm = 0.2716, lr_0 = 7.9788e-04
Validation prc-auc = 0.213847
Epoch 140
Validation prc-auc = 0.218795
Epoch 141
Validation prc-auc = 0.210443
Epoch 142
Validation prc-auc = 0.219385
Epoch 143
Validation prc-auc = 0.244349
Epoch 144
Loss = 1.2857e-02, PNorm = 85.9696, GNorm = 0.3995, lr_0 = 7.9106e-04
Validation prc-auc = 0.237443
Epoch 145
Validation prc-auc = 0.246441
Epoch 146
Validation prc-auc = 0.253707
Epoch 147
Validation prc-auc = 0.269453
Epoch 148
Validation prc-auc = 0.283431
Epoch 149
Loss = 8.9761e-03, PNorm = 86.1367, GNorm = 0.2976, lr_0 = 7.8430e-04
Validation prc-auc = 0.289997
Epoch 150
Validation prc-auc = 0.295357
Epoch 151
Validation prc-auc = 0.261798
Epoch 152
Validation prc-auc = 0.235727
Epoch 153
Validation prc-auc = 0.247289
Epoch 154
Loss = 1.5452e-02, PNorm = 86.2802, GNorm = 0.1277, lr_0 = 7.7759e-04
Validation prc-auc = 0.241072
Epoch 155
Validation prc-auc = 0.242246
Epoch 156
Validation prc-auc = 0.214299
Epoch 157
Validation prc-auc = 0.199380
Epoch 158
Validation prc-auc = 0.239951
Epoch 159
Loss = 5.9379e-03, PNorm = 86.4244, GNorm = 0.1018, lr_0 = 7.7095e-04
Validation prc-auc = 0.246812
Epoch 160
Validation prc-auc = 0.255179
Epoch 161
Validation prc-auc = 0.248417
Epoch 162
Validation prc-auc = 0.238683
Epoch 163
Validation prc-auc = 0.237195
Epoch 164
Loss = 1.9421e-03, PNorm = 86.5171, GNorm = 0.0091, lr_0 = 7.6436e-04
Validation prc-auc = 0.250266
Epoch 165
Validation prc-auc = 0.252963
Epoch 166
Validation prc-auc = 0.254370
Epoch 167
Validation prc-auc = 0.250415
Epoch 168
Validation prc-auc = 0.249695
Epoch 169
Loss = 1.3199e-03, PNorm = 86.5762, GNorm = 0.0447, lr_0 = 7.5782e-04
Validation prc-auc = 0.249349
Epoch 170
Validation prc-auc = 0.246538
Epoch 171
Validation prc-auc = 0.240236
Epoch 172
Validation prc-auc = 0.240534
Epoch 173
Validation prc-auc = 0.235577
Epoch 174
Loss = 2.2461e-03, PNorm = 86.6135, GNorm = 0.2446, lr_0 = 7.5134e-04
Validation prc-auc = 0.244731
Epoch 175
Validation prc-auc = 0.246985
Epoch 176
Validation prc-auc = 0.249017
Epoch 177
Validation prc-auc = 0.252572
Epoch 178
Validation prc-auc = 0.251968
Epoch 179
Loss = 6.3653e-04, PNorm = 86.6483, GNorm = 0.0177, lr_0 = 7.4492e-04
Validation prc-auc = 0.249646
Epoch 180
Validation prc-auc = 0.242470
Epoch 181
Validation prc-auc = 0.243353
Epoch 182
Validation prc-auc = 0.243127
Epoch 183
Validation prc-auc = 0.240641
Epoch 184
Loss = 4.6618e-04, PNorm = 86.6758, GNorm = 0.0062, lr_0 = 7.3855e-04
Validation prc-auc = 0.237657
Epoch 185
Validation prc-auc = 0.237527
Epoch 186
Validation prc-auc = 0.239330
Epoch 187
Validation prc-auc = 0.238970
Epoch 188
Validation prc-auc = 0.243761
Epoch 189
Loss = 1.4330e-03, PNorm = 86.6921, GNorm = 0.0315, lr_0 = 7.3224e-04
Validation prc-auc = 0.249133
Epoch 190
Validation prc-auc = 0.253101
Epoch 191
Validation prc-auc = 0.245300
Epoch 192
Validation prc-auc = 0.252355
Epoch 193
Validation prc-auc = 0.253956
Epoch 194
Loss = 3.3508e-04, PNorm = 86.7066, GNorm = 0.0125, lr_0 = 7.2598e-04
Validation prc-auc = 0.253320
Epoch 195
Validation prc-auc = 0.253719
Epoch 196
Validation prc-auc = 0.254532
Epoch 197
Validation prc-auc = 0.254145
Epoch 198
Validation prc-auc = 0.254142
Epoch 199
Loss = 3.3925e-04, PNorm = 86.7180, GNorm = 0.0175, lr_0 = 7.1977e-04
Validation prc-auc = 0.254220
Epoch 200
Validation prc-auc = 0.254019
Epoch 201
Validation prc-auc = 0.254220
Epoch 202
Validation prc-auc = 0.254220
Epoch 203
Validation prc-auc = 0.253649
Epoch 204
Loss = 1.5806e-04, PNorm = 86.7262, GNorm = 0.0059, lr_0 = 7.1362e-04
Validation prc-auc = 0.252956
Epoch 205
Validation prc-auc = 0.252673
Epoch 206
Validation prc-auc = 0.252604
Epoch 207
Validation prc-auc = 0.252604
Epoch 208
Validation prc-auc = 0.252525
Epoch 209
Loss = 8.6087e-05, PNorm = 86.7315, GNorm = 0.0030, lr_0 = 7.0752e-04
Validation prc-auc = 0.250358
Epoch 210
Validation prc-auc = 0.253926
Epoch 211
Validation prc-auc = 0.253731
Epoch 212
Validation prc-auc = 0.254005
Epoch 213
Validation prc-auc = 0.254205
Epoch 214
Loss = 2.0440e-04, PNorm = 86.7374, GNorm = 0.0069, lr_0 = 7.0147e-04
Validation prc-auc = 0.254903
Epoch 215
Validation prc-auc = 0.257509
Epoch 216
Validation prc-auc = 0.257265
Epoch 217
Validation prc-auc = 0.257896
Epoch 218
Validation prc-auc = 0.257896
Epoch 219
Loss = 1.3274e-04, PNorm = 86.7445, GNorm = 0.0034, lr_0 = 6.9548e-04
Validation prc-auc = 0.257900
Epoch 220
Validation prc-auc = 0.252493
Epoch 221
Validation prc-auc = 0.252493
Epoch 222
Validation prc-auc = 0.252430
Epoch 223
Validation prc-auc = 0.252665
Epoch 224
Loss = 2.3222e-04, PNorm = 86.7508, GNorm = 0.0105, lr_0 = 6.8953e-04
Validation prc-auc = 0.251845
Epoch 225
Validation prc-auc = 0.251845
Epoch 226
Validation prc-auc = 0.251988
Epoch 227
Validation prc-auc = 0.256996
Epoch 228
Validation prc-auc = 0.256713
Epoch 229
Loss = 1.0546e-04, PNorm = 86.7573, GNorm = 0.0036, lr_0 = 6.8364e-04
Validation prc-auc = 0.256713
Epoch 230
Validation prc-auc = 0.256467
Epoch 231
Validation prc-auc = 0.256467
Epoch 232
Validation prc-auc = 0.256197
Epoch 233
Validation prc-auc = 0.255886
Epoch 234
Loss = 1.0012e-04, PNorm = 86.7625, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.253234
Epoch 235
Validation prc-auc = 0.250456
Epoch 236
Validation prc-auc = 0.249570
Epoch 237
Validation prc-auc = 0.249646
Epoch 238
Validation prc-auc = 0.249646
Epoch 239
Loss = 1.4534e-04, PNorm = 86.7672, GNorm = 0.0069, lr_0 = 6.7200e-04
Validation prc-auc = 0.249527
Epoch 240
Validation prc-auc = 0.248775
Epoch 241
Validation prc-auc = 0.250517
Epoch 242
Validation prc-auc = 0.250440
Epoch 243
Validation prc-auc = 0.250516
Epoch 244
Loss = 1.7778e-04, PNorm = 86.7717, GNorm = 0.0076, lr_0 = 6.6625e-04
Validation prc-auc = 0.250440
Epoch 245
Validation prc-auc = 0.249983
Epoch 246
Validation prc-auc = 0.249986
Epoch 247
Validation prc-auc = 0.249986
Epoch 248
Validation prc-auc = 0.250038
Epoch 249
Loss = 5.5124e-05, PNorm = 86.7762, GNorm = 0.0019, lr_0 = 6.6056e-04
Validation prc-auc = 0.250105
Epoch 250
Validation prc-auc = 0.249858
Epoch 251
Validation prc-auc = 0.249911
Epoch 252
Validation prc-auc = 0.249723
Epoch 253
Validation prc-auc = 0.250603
Epoch 254
Loss = 6.4436e-05, PNorm = 86.7802, GNorm = 0.0013, lr_0 = 6.5491e-04
Validation prc-auc = 0.249911
Epoch 255
Validation prc-auc = 0.249990
Epoch 256
Validation prc-auc = 0.249803
Epoch 257
Validation prc-auc = 0.249567
Epoch 258
Validation prc-auc = 0.249567
Epoch 259
Loss = 9.6219e-05, PNorm = 86.7837, GNorm = 0.0085, lr_0 = 6.4931e-04
Validation prc-auc = 0.249567
Epoch 260
Validation prc-auc = 0.249635
Epoch 261
Validation prc-auc = 0.249822
Epoch 262
Validation prc-auc = 0.250218
Epoch 263
Validation prc-auc = 0.250218
Epoch 264
Loss = 5.8861e-05, PNorm = 86.7876, GNorm = 0.0022, lr_0 = 6.4376e-04
Validation prc-auc = 0.250150
Epoch 265
Validation prc-auc = 0.250150
Epoch 266
Validation prc-auc = 0.249923
Epoch 267
Validation prc-auc = 0.256431
Epoch 268
Validation prc-auc = 0.256215
Epoch 269
Loss = 1.0507e-04, PNorm = 86.7916, GNorm = 0.0065, lr_0 = 6.3826e-04
Validation prc-auc = 0.256013
Epoch 270
Validation prc-auc = 0.256090
Epoch 271
Validation prc-auc = 0.256090
Epoch 272
Validation prc-auc = 0.256090
Epoch 273
Validation prc-auc = 0.256090
Epoch 274
Loss = 5.5034e-05, PNorm = 86.7959, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.256090
Epoch 275
Validation prc-auc = 0.255896
Epoch 276
Validation prc-auc = 0.255819
Epoch 277
Validation prc-auc = 0.255611
Epoch 278
Validation prc-auc = 0.255611
Epoch 279
Loss = 7.1451e-05, PNorm = 86.8009, GNorm = 0.0042, lr_0 = 6.2739e-04
Validation prc-auc = 0.255611
Epoch 280
Validation prc-auc = 0.255597
Epoch 281
Validation prc-auc = 0.255597
Epoch 282
Validation prc-auc = 0.255397
Epoch 283
Validation prc-auc = 0.254757
Epoch 284
Loss = 1.0247e-04, PNorm = 86.8062, GNorm = 0.0051, lr_0 = 6.2203e-04
Validation prc-auc = 0.254957
Epoch 285
Validation prc-auc = 0.255151
Epoch 286
Validation prc-auc = 0.255353
Epoch 287
Validation prc-auc = 0.255629
Epoch 288
Validation prc-auc = 0.255629
Epoch 289
Loss = 5.7148e-05, PNorm = 86.8121, GNorm = 0.0029, lr_0 = 6.1671e-04
Validation prc-auc = 0.255706
Epoch 290
Validation prc-auc = 0.256346
Epoch 291
Validation prc-auc = 0.256346
Epoch 292
Validation prc-auc = 0.256063
Epoch 293
Validation prc-auc = 0.255600
Epoch 294
Loss = 5.4535e-05, PNorm = 86.8190, GNorm = 0.0005, lr_0 = 6.1144e-04
Validation prc-auc = 0.255335
Epoch 295
Validation prc-auc = 0.255335
Epoch 296
Validation prc-auc = 0.255335
Epoch 297
Validation prc-auc = 0.255143
Epoch 298
Validation prc-auc = 0.255143
Epoch 299
Loss = 4.1452e-05, PNorm = 86.8257, GNorm = 0.0015, lr_0 = 6.0621e-04
Validation prc-auc = 0.255216
Model 0 best validation prc-auc = 0.494079 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.473718
Ensemble test prc-auc = 0.473718
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.444433
Epoch 1
Validation prc-auc = 0.456129
Epoch 2
Validation prc-auc = 0.420647
Epoch 3
Validation prc-auc = 0.440192
Epoch 4
Loss = 6.5323e-01, PNorm = 78.5536, GNorm = 3.8553, lr_0 = 6.5000e-04
Validation prc-auc = 0.469897
Epoch 5
Validation prc-auc = 0.441008
Epoch 6
Validation prc-auc = 0.439744
Epoch 7
Validation prc-auc = 0.429229
Epoch 8
Validation prc-auc = 0.430780
Epoch 9
Loss = 6.5283e-01, PNorm = 78.7980, GNorm = 8.9798, lr_0 = 9.9743e-04
Validation prc-auc = 0.299335
Epoch 10
Validation prc-auc = 0.310533
Epoch 11
Validation prc-auc = 0.376633
Epoch 12
Validation prc-auc = 0.478044
Epoch 13
Validation prc-auc = 0.421652
Epoch 14
Loss = 5.3995e-01, PNorm = 79.1048, GNorm = 0.7729, lr_0 = 9.8890e-04
Validation prc-auc = 0.412349
Epoch 15
Validation prc-auc = 0.403937
Epoch 16
Validation prc-auc = 0.487608
Epoch 17
Validation prc-auc = 0.466166
Epoch 18
Validation prc-auc = 0.297784
Epoch 19
Loss = 4.1483e-01, PNorm = 79.4444, GNorm = 3.4478, lr_0 = 9.8045e-04
Validation prc-auc = 0.271431
Epoch 20
Validation prc-auc = 0.322065
Epoch 21
Validation prc-auc = 0.319643
Epoch 22
Validation prc-auc = 0.320843
Epoch 23
Validation prc-auc = 0.268977
Epoch 24
Loss = 5.0285e-01, PNorm = 79.8039, GNorm = 4.1674, lr_0 = 9.7207e-04
Validation prc-auc = 0.289349
Epoch 25
Validation prc-auc = 0.392423
Epoch 26
Validation prc-auc = 0.275621
Epoch 27
Validation prc-auc = 0.306595
Epoch 28
Validation prc-auc = 0.353579
Epoch 29
Loss = 4.1142e-01, PNorm = 80.2021, GNorm = 2.4667, lr_0 = 9.6376e-04
Validation prc-auc = 0.349656
Epoch 30
Validation prc-auc = 0.418446
Epoch 31
Validation prc-auc = 0.441640
Epoch 32
Validation prc-auc = 0.361011
Epoch 33
Validation prc-auc = 0.355823
Epoch 34
Loss = 3.6353e-01, PNorm = 80.5821, GNorm = 7.7298, lr_0 = 9.5552e-04
Validation prc-auc = 0.330606
Epoch 35
Validation prc-auc = 0.365648
Epoch 36
Validation prc-auc = 0.336102
Epoch 37
Validation prc-auc = 0.361568
Epoch 38
Validation prc-auc = 0.382436
Epoch 39
Loss = 2.5884e-01, PNorm = 81.0056, GNorm = 2.0205, lr_0 = 9.4735e-04
Validation prc-auc = 0.399115
Epoch 40
Validation prc-auc = 0.387232
Epoch 41
Validation prc-auc = 0.351931
Epoch 42
Validation prc-auc = 0.235007
Epoch 43
Validation prc-auc = 0.243459
Epoch 44
Loss = 1.6512e-01, PNorm = 81.4481, GNorm = 1.2873, lr_0 = 9.3925e-04
Validation prc-auc = 0.226750
Epoch 45
Validation prc-auc = 0.359088
Epoch 46
Validation prc-auc = 0.358515
Epoch 47
Validation prc-auc = 0.339205
Epoch 48
Validation prc-auc = 0.335852
Epoch 49
Loss = 1.2805e-01, PNorm = 81.9343, GNorm = 1.6719, lr_0 = 9.3122e-04
Validation prc-auc = 0.320263
Epoch 50
Validation prc-auc = 0.314936
Epoch 51
Validation prc-auc = 0.317689
Epoch 52
Validation prc-auc = 0.315497
Epoch 53
Validation prc-auc = 0.296828
Epoch 54
Loss = 8.5127e-02, PNorm = 82.3601, GNorm = 0.4456, lr_0 = 9.2326e-04
Validation prc-auc = 0.182812
Epoch 55
Validation prc-auc = 0.227757
Epoch 56
Validation prc-auc = 0.256377
Epoch 57
Validation prc-auc = 0.194120
Epoch 58
Validation prc-auc = 0.242637
Epoch 59
Loss = 7.3885e-02, PNorm = 82.7524, GNorm = 1.2944, lr_0 = 9.1537e-04
Validation prc-auc = 0.301696
Epoch 60
Validation prc-auc = 0.279037
Epoch 61
Validation prc-auc = 0.307725
Epoch 62
Validation prc-auc = 0.323613
Epoch 63
Validation prc-auc = 0.291489
Epoch 64
Loss = 4.3935e-02, PNorm = 83.1453, GNorm = 0.3841, lr_0 = 9.0754e-04
Validation prc-auc = 0.237511
Epoch 65
Validation prc-auc = 0.206105
Epoch 66
Validation prc-auc = 0.213431
Epoch 67
Validation prc-auc = 0.201107
Epoch 68
Validation prc-auc = 0.205966
Epoch 69
Loss = 3.8325e-02, PNorm = 83.5038, GNorm = 0.6145, lr_0 = 8.9978e-04
Validation prc-auc = 0.210441
Epoch 70
Validation prc-auc = 0.203410
Epoch 71
Validation prc-auc = 0.230668
Epoch 72
Validation prc-auc = 0.244779
Epoch 73
Validation prc-auc = 0.196868
Epoch 74
Loss = 1.0429e-01, PNorm = 83.8143, GNorm = 3.5757, lr_0 = 8.9209e-04
Validation prc-auc = 0.252263
Epoch 75
Validation prc-auc = 0.282624
Epoch 76
Validation prc-auc = 0.206495
Epoch 77
Validation prc-auc = 0.203314
Epoch 78
Validation prc-auc = 0.246712
Epoch 79
Loss = 2.2613e-02, PNorm = 84.1355, GNorm = 1.1337, lr_0 = 8.8447e-04
Validation prc-auc = 0.296338
Epoch 80
Validation prc-auc = 0.259639
Epoch 81
Validation prc-auc = 0.227287
Epoch 82
Validation prc-auc = 0.263878
Epoch 83
Validation prc-auc = 0.265886
Epoch 84
Loss = 1.8350e-02, PNorm = 84.4276, GNorm = 0.6847, lr_0 = 8.7691e-04
Validation prc-auc = 0.221338
Epoch 85
Validation prc-auc = 0.228542
Epoch 86
Validation prc-auc = 0.236924
Epoch 87
Validation prc-auc = 0.244105
Epoch 88
Validation prc-auc = 0.234104
Epoch 89
Loss = 4.6554e-02, PNorm = 84.6735, GNorm = 0.5354, lr_0 = 8.6941e-04
Validation prc-auc = 0.192770
Epoch 90
Validation prc-auc = 0.203480
Epoch 91
Validation prc-auc = 0.241460
Epoch 92
Validation prc-auc = 0.254533
Epoch 93
Validation prc-auc = 0.243593
Epoch 94
Loss = 3.3112e-02, PNorm = 84.9733, GNorm = 0.6421, lr_0 = 8.6198e-04
Validation prc-auc = 0.230764
Epoch 95
Validation prc-auc = 0.230331
Epoch 96
Validation prc-auc = 0.205606
Epoch 97
Validation prc-auc = 0.178229
Epoch 98
Validation prc-auc = 0.182735
Epoch 99
Loss = 1.0864e-02, PNorm = 85.2441, GNorm = 0.1917, lr_0 = 8.5461e-04
Validation prc-auc = 0.213917
Epoch 100
Validation prc-auc = 0.239236
Epoch 101
Validation prc-auc = 0.281919
Epoch 102
Validation prc-auc = 0.287679
Epoch 103
Validation prc-auc = 0.255842
Epoch 104
Loss = 9.8627e-03, PNorm = 85.4615, GNorm = 0.2517, lr_0 = 8.4730e-04
Validation prc-auc = 0.231141
Epoch 105
Validation prc-auc = 0.227071
Epoch 106
Validation prc-auc = 0.257809
Epoch 107
Validation prc-auc = 0.267455
Epoch 108
Validation prc-auc = 0.277898
Epoch 109
Loss = 4.2696e-03, PNorm = 85.6242, GNorm = 0.1194, lr_0 = 8.4006e-04
Validation prc-auc = 0.281482
Epoch 110
Validation prc-auc = 0.328216
Epoch 111
Validation prc-auc = 0.310008
Epoch 112
Validation prc-auc = 0.267946
Epoch 113
Validation prc-auc = 0.247763
Epoch 114
Loss = 1.0243e-02, PNorm = 85.7935, GNorm = 0.4677, lr_0 = 8.3288e-04
Validation prc-auc = 0.261461
Epoch 115
Validation prc-auc = 0.242210
Epoch 116
Validation prc-auc = 0.242133
Epoch 117
Validation prc-auc = 0.276419
Epoch 118
Validation prc-auc = 0.200080
Epoch 119
Loss = 7.4002e-02, PNorm = 86.1586, GNorm = 2.8840, lr_0 = 8.2576e-04
Validation prc-auc = 0.180384
Epoch 120
Validation prc-auc = 0.285562
Epoch 121
Validation prc-auc = 0.311209
Epoch 122
Validation prc-auc = 0.294940
Epoch 123
Validation prc-auc = 0.249885
Epoch 124
Loss = 1.7930e-01, PNorm = 86.5154, GNorm = 0.8035, lr_0 = 8.1870e-04
Validation prc-auc = 0.255555
Epoch 125
Validation prc-auc = 0.230887
Epoch 126
Validation prc-auc = 0.228909
Epoch 127
Validation prc-auc = 0.264813
Epoch 128
Validation prc-auc = 0.275429
Epoch 129
Loss = 1.8357e-01, PNorm = 86.8617, GNorm = 4.5801, lr_0 = 8.1170e-04
Validation prc-auc = 0.229094
Epoch 130
Validation prc-auc = 0.194775
Epoch 131
Validation prc-auc = 0.168031
Epoch 132
Validation prc-auc = 0.191172
Epoch 133
Validation prc-auc = 0.227478
Epoch 134
Loss = 1.1342e-02, PNorm = 87.2355, GNorm = 0.2556, lr_0 = 8.0476e-04
Validation prc-auc = 0.246038
Epoch 135
Validation prc-auc = 0.255616
Epoch 136
Validation prc-auc = 0.254731
Epoch 137
Validation prc-auc = 0.249726
Epoch 138
Validation prc-auc = 0.245840
Epoch 139
Loss = 6.2263e-03, PNorm = 87.4927, GNorm = 0.0195, lr_0 = 7.9788e-04
Validation prc-auc = 0.244570
Epoch 140
Validation prc-auc = 0.235615
Epoch 141
Validation prc-auc = 0.220272
Epoch 142
Validation prc-auc = 0.208459
Epoch 143
Validation prc-auc = 0.200123
Epoch 144
Loss = 4.0279e-03, PNorm = 87.6342, GNorm = 0.1075, lr_0 = 7.9106e-04
Validation prc-auc = 0.192568
Epoch 145
Validation prc-auc = 0.186821
Epoch 146
Validation prc-auc = 0.184707
Epoch 147
Validation prc-auc = 0.195172
Epoch 148
Validation prc-auc = 0.213246
Epoch 149
Loss = 3.8667e-03, PNorm = 87.7297, GNorm = 0.1326, lr_0 = 7.8430e-04
Validation prc-auc = 0.215438
Epoch 150
Validation prc-auc = 0.202191
Epoch 151
Validation prc-auc = 0.202999
Epoch 152
Validation prc-auc = 0.195512
Epoch 153
Validation prc-auc = 0.212758
Epoch 154
Loss = 3.2520e-03, PNorm = 87.8064, GNorm = 0.0373, lr_0 = 7.7759e-04
Validation prc-auc = 0.237215
Epoch 155
Validation prc-auc = 0.244448
Epoch 156
Validation prc-auc = 0.251906
Epoch 157
Validation prc-auc = 0.237463
Epoch 158
Validation prc-auc = 0.221265
Epoch 159
Loss = 1.5139e-03, PNorm = 87.8657, GNorm = 0.0233, lr_0 = 7.7095e-04
Validation prc-auc = 0.219616
Epoch 160
Validation prc-auc = 0.210079
Epoch 161
Validation prc-auc = 0.208258
Epoch 162
Validation prc-auc = 0.207564
Epoch 163
Validation prc-auc = 0.211004
Epoch 164
Loss = 2.6689e-03, PNorm = 87.9153, GNorm = 0.0170, lr_0 = 7.6436e-04
Validation prc-auc = 0.215004
Epoch 165
Validation prc-auc = 0.227429
Epoch 166
Validation prc-auc = 0.244716
Epoch 167
Validation prc-auc = 0.243009
Epoch 168
Validation prc-auc = 0.246214
Epoch 169
Loss = 1.8959e-03, PNorm = 87.9682, GNorm = 0.0144, lr_0 = 7.5782e-04
Validation prc-auc = 0.244058
Epoch 170
Validation prc-auc = 0.246435
Epoch 171
Validation prc-auc = 0.237189
Epoch 172
Validation prc-auc = 0.231434
Epoch 173
Validation prc-auc = 0.230957
Epoch 174
Loss = 3.2964e-03, PNorm = 88.0205, GNorm = 0.2178, lr_0 = 7.5134e-04
Validation prc-auc = 0.230271
Epoch 175
Validation prc-auc = 0.236124
Epoch 176
Validation prc-auc = 0.234886
Epoch 177
Validation prc-auc = 0.233873
Epoch 178
Validation prc-auc = 0.238762
Epoch 179
Loss = 9.4037e-04, PNorm = 88.0521, GNorm = 0.0239, lr_0 = 7.4492e-04
Validation prc-auc = 0.240527
Epoch 180
Validation prc-auc = 0.241138
Epoch 181
Validation prc-auc = 0.238063
Epoch 182
Validation prc-auc = 0.238238
Epoch 183
Validation prc-auc = 0.234292
Epoch 184
Loss = 4.1477e-04, PNorm = 88.0769, GNorm = 0.0094, lr_0 = 7.3855e-04
Validation prc-auc = 0.234197
Epoch 185
Validation prc-auc = 0.234250
Epoch 186
Validation prc-auc = 0.233476
Epoch 187
Validation prc-auc = 0.233148
Epoch 188
Validation prc-auc = 0.233055
Epoch 189
Loss = 6.4001e-04, PNorm = 88.0948, GNorm = 0.0178, lr_0 = 7.3224e-04
Validation prc-auc = 0.232867
Epoch 190
Validation prc-auc = 0.233167
Epoch 191
Validation prc-auc = 0.233055
Epoch 192
Validation prc-auc = 0.236434
Epoch 193
Validation prc-auc = 0.236434
Epoch 194
Loss = 3.1285e-04, PNorm = 88.1079, GNorm = 0.0107, lr_0 = 7.2598e-04
Validation prc-auc = 0.236260
Epoch 195
Validation prc-auc = 0.235655
Epoch 196
Validation prc-auc = 0.236170
Epoch 197
Validation prc-auc = 0.239206
Epoch 198
Validation prc-auc = 0.239687
Epoch 199
Loss = 1.6953e-04, PNorm = 88.1196, GNorm = 0.0048, lr_0 = 7.1977e-04
Validation prc-auc = 0.244501
Epoch 200
Validation prc-auc = 0.244501
Epoch 201
Validation prc-auc = 0.244448
Epoch 202
Validation prc-auc = 0.244448
Epoch 203
Validation prc-auc = 0.244448
Epoch 204
Loss = 1.7062e-04, PNorm = 88.1297, GNorm = 0.0036, lr_0 = 7.1362e-04
Validation prc-auc = 0.248064
Epoch 205
Validation prc-auc = 0.248064
Epoch 206
Validation prc-auc = 0.248155
Epoch 207
Validation prc-auc = 0.243041
Epoch 208
Validation prc-auc = 0.243115
Epoch 209
Loss = 3.0831e-04, PNorm = 88.1389, GNorm = 0.0104, lr_0 = 7.0752e-04
Validation prc-auc = 0.243065
Epoch 210
Validation prc-auc = 0.243065
Epoch 211
Validation prc-auc = 0.242946
Epoch 212
Validation prc-auc = 0.242946
Epoch 213
Validation prc-auc = 0.243838
Epoch 214
Loss = 3.3854e-04, PNorm = 88.1500, GNorm = 0.0195, lr_0 = 7.0147e-04
Validation prc-auc = 0.243941
Epoch 215
Validation prc-auc = 0.243829
Epoch 216
Validation prc-auc = 0.243689
Epoch 217
Validation prc-auc = 0.243780
Epoch 218
Validation prc-auc = 0.243670
Epoch 219
Loss = 1.2852e-04, PNorm = 88.1621, GNorm = 0.0031, lr_0 = 6.9548e-04
Validation prc-auc = 0.243670
Epoch 220
Validation prc-auc = 0.243670
Epoch 221
Validation prc-auc = 0.243344
Epoch 222
Validation prc-auc = 0.243896
Epoch 223
Validation prc-auc = 0.247709
Epoch 224
Loss = 9.1210e-05, PNorm = 88.1722, GNorm = 0.0038, lr_0 = 6.8953e-04
Validation prc-auc = 0.247617
Epoch 225
Validation prc-auc = 0.247228
Epoch 226
Validation prc-auc = 0.247228
Epoch 227
Validation prc-auc = 0.247391
Epoch 228
Validation prc-auc = 0.247207
Epoch 229
Loss = 1.6087e-04, PNorm = 88.1795, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.247207
Epoch 230
Validation prc-auc = 0.243668
Epoch 231
Validation prc-auc = 0.243751
Epoch 232
Validation prc-auc = 0.243055
Epoch 233
Validation prc-auc = 0.243023
Epoch 234
Loss = 3.0499e-04, PNorm = 88.1857, GNorm = 0.0040, lr_0 = 6.7779e-04
Validation prc-auc = 0.242620
Epoch 235
Validation prc-auc = 0.242620
Epoch 236
Validation prc-auc = 0.246080
Epoch 237
Validation prc-auc = 0.245971
Epoch 238
Validation prc-auc = 0.246027
Epoch 239
Loss = 1.0634e-04, PNorm = 88.1956, GNorm = 0.0032, lr_0 = 6.7200e-04
Validation prc-auc = 0.246119
Epoch 240
Validation prc-auc = 0.246074
Epoch 241
Validation prc-auc = 0.246182
Epoch 242
Validation prc-auc = 0.246800
Epoch 243
Validation prc-auc = 0.246733
Epoch 244
Loss = 7.6656e-05, PNorm = 88.2041, GNorm = 0.0031, lr_0 = 6.6625e-04
Validation prc-auc = 0.246733
Epoch 245
Validation prc-auc = 0.250039
Epoch 246
Validation prc-auc = 0.250039
Epoch 247
Validation prc-auc = 0.250039
Epoch 248
Validation prc-auc = 0.252690
Epoch 249
Loss = 8.8182e-05, PNorm = 88.2116, GNorm = 0.0031, lr_0 = 6.6056e-04
Validation prc-auc = 0.252536
Epoch 250
Validation prc-auc = 0.252603
Epoch 251
Validation prc-auc = 0.252820
Epoch 252
Validation prc-auc = 0.252596
Epoch 253
Validation prc-auc = 0.252877
Epoch 254
Loss = 1.1104e-04, PNorm = 88.2205, GNorm = 0.0021, lr_0 = 6.5491e-04
Validation prc-auc = 0.252811
Epoch 255
Validation prc-auc = 0.252208
Epoch 256
Validation prc-auc = 0.252420
Epoch 257
Validation prc-auc = 0.251804
Epoch 258
Validation prc-auc = 0.251741
Epoch 259
Loss = 8.3043e-05, PNorm = 88.2293, GNorm = 0.0040, lr_0 = 6.4931e-04
Validation prc-auc = 0.251280
Epoch 260
Validation prc-auc = 0.250420
Epoch 261
Validation prc-auc = 0.250420
Epoch 262
Validation prc-auc = 0.250327
Epoch 263
Validation prc-auc = 0.250420
Epoch 264
Loss = 4.5222e-05, PNorm = 88.2363, GNorm = 0.0010, lr_0 = 6.4376e-04
Validation prc-auc = 0.250901
Epoch 265
Validation prc-auc = 0.251021
Epoch 266
Validation prc-auc = 0.250712
Epoch 267
Validation prc-auc = 0.250231
Epoch 268
Validation prc-auc = 0.250298
Epoch 269
Loss = 4.5276e-05, PNorm = 88.2420, GNorm = 0.0015, lr_0 = 6.3826e-04
Validation prc-auc = 0.254069
Epoch 270
Validation prc-auc = 0.253805
Epoch 271
Validation prc-auc = 0.253968
Epoch 272
Validation prc-auc = 0.253546
Epoch 273
Validation prc-auc = 0.253968
Epoch 274
Loss = 5.4851e-05, PNorm = 88.2473, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.254352
Epoch 275
Validation prc-auc = 0.254282
Epoch 276
Validation prc-auc = 0.254763
Epoch 277
Validation prc-auc = 0.254830
Epoch 278
Validation prc-auc = 0.254807
Epoch 279
Loss = 3.4437e-05, PNorm = 88.2520, GNorm = 0.0017, lr_0 = 6.2739e-04
Validation prc-auc = 0.255005
Epoch 280
Validation prc-auc = 0.255532
Epoch 281
Validation prc-auc = 0.255532
Epoch 282
Validation prc-auc = 0.255348
Epoch 283
Validation prc-auc = 0.255501
Epoch 284
Loss = 6.7237e-05, PNorm = 88.2572, GNorm = 0.0009, lr_0 = 6.2203e-04
Validation prc-auc = 0.255957
Epoch 285
Validation prc-auc = 0.256053
Epoch 286
Validation prc-auc = 0.260197
Epoch 287
Validation prc-auc = 0.260197
Epoch 288
Validation prc-auc = 0.266863
Epoch 289
Loss = 2.3074e-05, PNorm = 88.2622, GNorm = 0.0009, lr_0 = 6.1671e-04
Validation prc-auc = 0.266264
Epoch 290
Validation prc-auc = 0.266060
Epoch 291
Validation prc-auc = 0.266046
Epoch 292
Validation prc-auc = 0.265635
Epoch 293
Validation prc-auc = 0.265446
Epoch 294
Loss = 3.0786e-05, PNorm = 88.2662, GNorm = 0.0002, lr_0 = 6.1144e-04
Validation prc-auc = 0.265517
Epoch 295
Validation prc-auc = 0.265998
Epoch 296
Validation prc-auc = 0.266091
Epoch 297
Validation prc-auc = 0.266091
Epoch 298
Validation prc-auc = 0.266174
Epoch 299
Loss = 3.7240e-05, PNorm = 88.2701, GNorm = 0.0037, lr_0 = 6.0621e-04
Validation prc-auc = 0.266137
Model 0 best validation prc-auc = 0.487608 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.528662
Ensemble test prc-auc = 0.528662
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.420661
Epoch 1
Validation prc-auc = 0.457936
Epoch 2
Validation prc-auc = 0.458228
Epoch 3
Validation prc-auc = 0.461219
Epoch 4
Loss = 6.7879e-01, PNorm = 78.5484, GNorm = 9.4250, lr_0 = 6.5000e-04
Validation prc-auc = 0.457434
Epoch 5
Validation prc-auc = 0.485822
Epoch 6
Validation prc-auc = 0.469495
Epoch 7
Validation prc-auc = 0.474223
Epoch 8
Validation prc-auc = 0.490912
Epoch 9
Loss = 6.7939e-01, PNorm = 78.7767, GNorm = 12.4080, lr_0 = 9.9743e-04
Validation prc-auc = 0.337378
Epoch 10
Validation prc-auc = 0.301258
Epoch 11
Validation prc-auc = 0.259962
Epoch 12
Validation prc-auc = 0.370990
Epoch 13
Validation prc-auc = 0.403101
Epoch 14
Loss = 6.1304e-01, PNorm = 79.0637, GNorm = 0.8893, lr_0 = 9.8890e-04
Validation prc-auc = 0.309827
Epoch 15
Validation prc-auc = 0.291735
Epoch 16
Validation prc-auc = 0.307380
Epoch 17
Validation prc-auc = 0.308140
Epoch 18
Validation prc-auc = 0.428703
Epoch 19
Loss = 5.2677e-01, PNorm = 79.3599, GNorm = 2.3717, lr_0 = 9.8045e-04
Validation prc-auc = 0.404927
Epoch 20
Validation prc-auc = 0.361680
Epoch 21
Validation prc-auc = 0.450644
Epoch 22
Validation prc-auc = 0.426842
Epoch 23
Validation prc-auc = 0.392685
Epoch 24
Loss = 4.6043e-01, PNorm = 79.6595, GNorm = 4.8828, lr_0 = 9.7207e-04
Validation prc-auc = 0.349631
Epoch 25
Validation prc-auc = 0.326684
Epoch 26
Validation prc-auc = 0.319028
Epoch 27
Validation prc-auc = 0.348372
Epoch 28
Validation prc-auc = 0.288309
Epoch 29
Loss = 3.7091e-01, PNorm = 79.9930, GNorm = 1.4221, lr_0 = 9.6376e-04
Validation prc-auc = 0.297019
Epoch 30
Validation prc-auc = 0.285389
Epoch 31
Validation prc-auc = 0.314174
Epoch 32
Validation prc-auc = 0.328227
Epoch 33
Validation prc-auc = 0.425654
Epoch 34
Loss = 3.2539e-01, PNorm = 80.3645, GNorm = 2.5136, lr_0 = 9.5552e-04
Validation prc-auc = 0.451703
Epoch 35
Validation prc-auc = 0.439275
Epoch 36
Validation prc-auc = 0.344190
Epoch 37
Validation prc-auc = 0.233892
Epoch 38
Validation prc-auc = 0.332029
Epoch 39
Loss = 2.5436e-01, PNorm = 80.7576, GNorm = 4.8816, lr_0 = 9.4735e-04
Validation prc-auc = 0.360506
Epoch 40
Validation prc-auc = 0.365633
Epoch 41
Validation prc-auc = 0.363018
Epoch 42
Validation prc-auc = 0.362571
Epoch 43
Validation prc-auc = 0.280235
Epoch 44
Loss = 2.9132e-01, PNorm = 81.2063, GNorm = 4.9521, lr_0 = 9.3925e-04
Validation prc-auc = 0.313984
Epoch 45
Validation prc-auc = 0.341950
Epoch 46
Validation prc-auc = 0.338457
Epoch 47
Validation prc-auc = 0.309914
Epoch 48
Validation prc-auc = 0.190773
Epoch 49
Loss = 2.0441e-01, PNorm = 81.7002, GNorm = 1.1314, lr_0 = 9.3122e-04
Validation prc-auc = 0.162704
Epoch 50
Validation prc-auc = 0.248556
Epoch 51
Validation prc-auc = 0.297140
Epoch 52
Validation prc-auc = 0.301968
Epoch 53
Validation prc-auc = 0.312886
Epoch 54
Loss = 1.0181e-01, PNorm = 82.1900, GNorm = 0.4953, lr_0 = 9.2326e-04
Validation prc-auc = 0.186690
Epoch 55
Validation prc-auc = 0.328122
Epoch 56
Validation prc-auc = 0.225372
Epoch 57
Validation prc-auc = 0.280809
Epoch 58
Validation prc-auc = 0.329574
Epoch 59
Loss = 3.3973e-01, PNorm = 82.6580, GNorm = 4.7337, lr_0 = 9.1537e-04
Validation prc-auc = 0.346119
Epoch 60
Validation prc-auc = 0.279217
Epoch 61
Validation prc-auc = 0.288589
Epoch 62
Validation prc-auc = 0.334008
Epoch 63
Validation prc-auc = 0.298605
Epoch 64
Loss = 8.9413e-02, PNorm = 83.0737, GNorm = 0.3650, lr_0 = 9.0754e-04
Validation prc-auc = 0.307533
Epoch 65
Validation prc-auc = 0.287666
Epoch 66
Validation prc-auc = 0.270246
Epoch 67
Validation prc-auc = 0.226745
Epoch 68
Validation prc-auc = 0.221687
Epoch 69
Loss = 7.1962e-02, PNorm = 83.4600, GNorm = 1.0944, lr_0 = 8.9978e-04
Validation prc-auc = 0.226417
Epoch 70
Validation prc-auc = 0.256125
Epoch 71
Validation prc-auc = 0.272778
Epoch 72
Validation prc-auc = 0.247839
Epoch 73
Validation prc-auc = 0.248686
Epoch 74
Loss = 6.4427e-02, PNorm = 83.7716, GNorm = 0.3512, lr_0 = 8.9209e-04
Validation prc-auc = 0.311930
Epoch 75
Validation prc-auc = 0.320713
Epoch 76
Validation prc-auc = 0.306418
Epoch 77
Validation prc-auc = 0.293195
Epoch 78
Validation prc-auc = 0.283546
Epoch 79
Loss = 5.7602e-02, PNorm = 84.0756, GNorm = 0.7084, lr_0 = 8.8447e-04
Validation prc-auc = 0.250902
Epoch 80
Validation prc-auc = 0.233777
Epoch 81
Validation prc-auc = 0.224770
Epoch 82
Validation prc-auc = 0.240630
Epoch 83
Validation prc-auc = 0.257950
Epoch 84
Loss = 2.8243e-02, PNorm = 84.3772, GNorm = 0.0990, lr_0 = 8.7691e-04
Validation prc-auc = 0.239297
Epoch 85
Validation prc-auc = 0.275845
Epoch 86
Validation prc-auc = 0.296512
Epoch 87
Validation prc-auc = 0.275751
Epoch 88
Validation prc-auc = 0.281087
Epoch 89
Loss = 1.3489e-02, PNorm = 84.6147, GNorm = 0.8700, lr_0 = 8.6941e-04
Validation prc-auc = 0.300239
Epoch 90
Validation prc-auc = 0.307408
Epoch 91
Validation prc-auc = 0.305317
Epoch 92
Validation prc-auc = 0.283052
Epoch 93
Validation prc-auc = 0.263338
Epoch 94
Loss = 4.1247e-02, PNorm = 84.8298, GNorm = 0.7225, lr_0 = 8.6198e-04
Validation prc-auc = 0.242708
Epoch 95
Validation prc-auc = 0.217276
Epoch 96
Validation prc-auc = 0.243376
Epoch 97
Validation prc-auc = 0.294436
Epoch 98
Validation prc-auc = 0.307211
Epoch 99
Loss = 1.7993e-02, PNorm = 85.0559, GNorm = 0.5066, lr_0 = 8.5461e-04
Validation prc-auc = 0.292837
Epoch 100
Validation prc-auc = 0.282350
Epoch 101
Validation prc-auc = 0.246013
Epoch 102
Validation prc-auc = 0.248986
Epoch 103
Validation prc-auc = 0.263404
Epoch 104
Loss = 3.3377e-02, PNorm = 85.2747, GNorm = 0.1569, lr_0 = 8.4730e-04
Validation prc-auc = 0.336663
Epoch 105
Validation prc-auc = 0.346621
Epoch 106
Validation prc-auc = 0.342681
Epoch 107
Validation prc-auc = 0.334539
Epoch 108
Validation prc-auc = 0.328910
Epoch 109
Loss = 2.5633e-02, PNorm = 85.4632, GNorm = 1.8693, lr_0 = 8.4006e-04
Validation prc-auc = 0.302107
Epoch 110
Validation prc-auc = 0.211599
Epoch 111
Validation prc-auc = 0.264865
Epoch 112
Validation prc-auc = 0.275362
Epoch 113
Validation prc-auc = 0.229602
Epoch 114
Loss = 6.8220e-02, PNorm = 85.6689, GNorm = 2.2773, lr_0 = 8.3288e-04
Validation prc-auc = 0.275315
Epoch 115
Validation prc-auc = 0.304039
Epoch 116
Validation prc-auc = 0.255938
Epoch 117
Validation prc-auc = 0.233755
Epoch 118
Validation prc-auc = 0.216920
Epoch 119
Loss = 6.1016e-02, PNorm = 85.9954, GNorm = 0.2323, lr_0 = 8.2576e-04
Validation prc-auc = 0.234682
Epoch 120
Validation prc-auc = 0.222558
Epoch 121
Validation prc-auc = 0.219669
Epoch 122
Validation prc-auc = 0.237908
Epoch 123
Validation prc-auc = 0.228341
Epoch 124
Loss = 2.3777e-02, PNorm = 86.2930, GNorm = 0.1455, lr_0 = 8.1870e-04
Validation prc-auc = 0.210794
Epoch 125
Validation prc-auc = 0.199932
Epoch 126
Validation prc-auc = 0.221785
Epoch 127
Validation prc-auc = 0.228188
Epoch 128
Validation prc-auc = 0.229822
Epoch 129
Loss = 2.1434e-02, PNorm = 86.5291, GNorm = 0.5616, lr_0 = 8.1170e-04
Validation prc-auc = 0.243109
Epoch 130
Validation prc-auc = 0.236189
Epoch 131
Validation prc-auc = 0.207302
Epoch 132
Validation prc-auc = 0.193973
Epoch 133
Validation prc-auc = 0.217384
Epoch 134
Loss = 1.2687e-01, PNorm = 86.7778, GNorm = 2.5584, lr_0 = 8.0476e-04
Validation prc-auc = 0.221427
Epoch 135
Validation prc-auc = 0.225822
Epoch 136
Validation prc-auc = 0.221941
Epoch 137
Validation prc-auc = 0.224318
Epoch 138
Validation prc-auc = 0.235118
Epoch 139
Loss = 1.2207e-02, PNorm = 86.9729, GNorm = 0.0883, lr_0 = 7.9788e-04
Validation prc-auc = 0.241458
Epoch 140
Validation prc-auc = 0.247542
Epoch 141
Validation prc-auc = 0.255155
Epoch 142
Validation prc-auc = 0.255968
Epoch 143
Validation prc-auc = 0.274385
Epoch 144
Loss = 3.4209e-02, PNorm = 87.1143, GNorm = 0.5921, lr_0 = 7.9106e-04
Validation prc-auc = 0.285068
Epoch 145
Validation prc-auc = 0.279809
Epoch 146
Validation prc-auc = 0.269946
Epoch 147
Validation prc-auc = 0.262443
Epoch 148
Validation prc-auc = 0.239521
Epoch 149
Loss = 7.4511e-03, PNorm = 87.2699, GNorm = 0.1724, lr_0 = 7.8430e-04
Validation prc-auc = 0.227633
Epoch 150
Validation prc-auc = 0.218389
Epoch 151
Validation prc-auc = 0.214915
Epoch 152
Validation prc-auc = 0.260338
Epoch 153
Validation prc-auc = 0.270727
Epoch 154
Loss = 4.3829e-03, PNorm = 87.4757, GNorm = 0.0986, lr_0 = 7.7759e-04
Validation prc-auc = 0.248127
Epoch 155
Validation prc-auc = 0.271496
Epoch 156
Validation prc-auc = 0.279499
Epoch 157
Validation prc-auc = 0.273700
Epoch 158
Validation prc-auc = 0.233976
Epoch 159
Loss = 3.6222e-02, PNorm = 87.6901, GNorm = 0.5669, lr_0 = 7.7095e-04
Validation prc-auc = 0.181578
Epoch 160
Validation prc-auc = 0.197170
Epoch 161
Validation prc-auc = 0.211570
Epoch 162
Validation prc-auc = 0.206212
Epoch 163
Validation prc-auc = 0.215330
Epoch 164
Loss = 1.2465e-01, PNorm = 87.9610, GNorm = 2.2373, lr_0 = 7.6436e-04
Validation prc-auc = 0.265177
Epoch 165
Validation prc-auc = 0.274934
Epoch 166
Validation prc-auc = 0.231637
Epoch 167
Validation prc-auc = 0.204012
Epoch 168
Validation prc-auc = 0.189641
Epoch 169
Loss = 3.4128e-02, PNorm = 88.2434, GNorm = 0.3778, lr_0 = 7.5782e-04
Validation prc-auc = 0.208085
Epoch 170
Validation prc-auc = 0.257314
Epoch 171
Validation prc-auc = 0.319175
Epoch 172
Validation prc-auc = 0.315239
Epoch 173
Validation prc-auc = 0.280051
Epoch 174
Loss = 6.4917e-03, PNorm = 88.5055, GNorm = 0.1110, lr_0 = 7.5134e-04
Validation prc-auc = 0.211732
Epoch 175
Validation prc-auc = 0.184104
Epoch 176
Validation prc-auc = 0.207875
Epoch 177
Validation prc-auc = 0.237450
Epoch 178
Validation prc-auc = 0.266153
Epoch 179
Loss = 6.7845e-03, PNorm = 88.6905, GNorm = 0.0565, lr_0 = 7.4492e-04
Validation prc-auc = 0.292587
Epoch 180
Validation prc-auc = 0.296856
Epoch 181
Validation prc-auc = 0.306850
Epoch 182
Validation prc-auc = 0.286291
Epoch 183
Validation prc-auc = 0.282767
Epoch 184
Loss = 3.5275e-03, PNorm = 88.8265, GNorm = 0.0604, lr_0 = 7.3855e-04
Validation prc-auc = 0.268273
Epoch 185
Validation prc-auc = 0.266838
Epoch 186
Validation prc-auc = 0.263637
Epoch 187
Validation prc-auc = 0.261352
Epoch 188
Validation prc-auc = 0.267484
Epoch 189
Loss = 2.5836e-03, PNorm = 88.9277, GNorm = 0.0224, lr_0 = 7.3224e-04
Validation prc-auc = 0.267054
Epoch 190
Validation prc-auc = 0.268399
Epoch 191
Validation prc-auc = 0.276735
Epoch 192
Validation prc-auc = 0.287725
Epoch 193
Validation prc-auc = 0.288679
Epoch 194
Loss = 1.9950e-03, PNorm = 88.9891, GNorm = 0.0316, lr_0 = 7.2598e-04
Validation prc-auc = 0.289114
Epoch 195
Validation prc-auc = 0.288929
Epoch 196
Validation prc-auc = 0.282456
Epoch 197
Validation prc-auc = 0.272728
Epoch 198
Validation prc-auc = 0.271680
Epoch 199
Loss = 8.0198e-04, PNorm = 89.0349, GNorm = 0.0207, lr_0 = 7.1977e-04
Validation prc-auc = 0.271598
Epoch 200
Validation prc-auc = 0.271760
Epoch 201
Validation prc-auc = 0.271311
Epoch 202
Validation prc-auc = 0.271315
Epoch 203
Validation prc-auc = 0.271311
Epoch 204
Loss = 1.0285e-03, PNorm = 89.0619, GNorm = 0.1913, lr_0 = 7.1362e-04
Validation prc-auc = 0.271392
Epoch 205
Validation prc-auc = 0.278023
Epoch 206
Validation prc-auc = 0.281764
Epoch 207
Validation prc-auc = 0.281656
Epoch 208
Validation prc-auc = 0.283690
Epoch 209
Loss = 1.4612e-03, PNorm = 89.0858, GNorm = 0.0122, lr_0 = 7.0752e-04
Validation prc-auc = 0.283638
Epoch 210
Validation prc-auc = 0.283437
Epoch 211
Validation prc-auc = 0.281394
Epoch 212
Validation prc-auc = 0.279675
Epoch 213
Validation prc-auc = 0.279458
Epoch 214
Loss = 4.5339e-04, PNorm = 89.1154, GNorm = 0.0102, lr_0 = 7.0147e-04
Validation prc-auc = 0.279104
Epoch 215
Validation prc-auc = 0.279294
Epoch 216
Validation prc-auc = 0.277534
Epoch 217
Validation prc-auc = 0.277420
Epoch 218
Validation prc-auc = 0.277356
Epoch 219
Loss = 1.2988e-03, PNorm = 89.1366, GNorm = 0.0905, lr_0 = 6.9548e-04
Validation prc-auc = 0.277290
Epoch 220
Validation prc-auc = 0.277382
Epoch 221
Validation prc-auc = 0.277386
Epoch 222
Validation prc-auc = 0.277469
Epoch 223
Validation prc-auc = 0.277310
Epoch 224
Loss = 2.4854e-04, PNorm = 89.1491, GNorm = 0.0055, lr_0 = 6.8953e-04
Validation prc-auc = 0.277302
Epoch 225
Validation prc-auc = 0.277302
Epoch 226
Validation prc-auc = 0.277252
Epoch 227
Validation prc-auc = 0.277247
Epoch 228
Validation prc-auc = 0.277420
Epoch 229
Loss = 1.6646e-04, PNorm = 89.1637, GNorm = 0.0050, lr_0 = 6.8364e-04
Validation prc-auc = 0.277234
Epoch 230
Validation prc-auc = 0.277172
Epoch 231
Validation prc-auc = 0.277155
Epoch 232
Validation prc-auc = 0.277040
Epoch 233
Validation prc-auc = 0.283128
Epoch 234
Loss = 1.7735e-04, PNorm = 89.1747, GNorm = 0.0067, lr_0 = 6.7779e-04
Validation prc-auc = 0.283067
Epoch 235
Validation prc-auc = 0.283079
Epoch 236
Validation prc-auc = 0.283030
Epoch 237
Validation prc-auc = 0.281935
Epoch 238
Validation prc-auc = 0.281935
Epoch 239
Loss = 1.2403e-04, PNorm = 89.1835, GNorm = 0.0077, lr_0 = 6.7200e-04
Validation prc-auc = 0.281935
Epoch 240
Validation prc-auc = 0.281935
Epoch 241
Validation prc-auc = 0.281935
Epoch 242
Validation prc-auc = 0.282439
Epoch 243
Validation prc-auc = 0.282439
Epoch 244
Loss = 1.6262e-04, PNorm = 89.1912, GNorm = 0.0020, lr_0 = 6.6625e-04
Validation prc-auc = 0.282439
Epoch 245
Validation prc-auc = 0.282390
Epoch 246
Validation prc-auc = 0.282390
Epoch 247
Validation prc-auc = 0.282342
Epoch 248
Validation prc-auc = 0.282403
Epoch 249
Loss = 1.6689e-04, PNorm = 89.1992, GNorm = 0.0021, lr_0 = 6.6056e-04
Validation prc-auc = 0.282403
Epoch 250
Validation prc-auc = 0.282470
Epoch 251
Validation prc-auc = 0.282424
Epoch 252
Validation prc-auc = 0.282487
Epoch 253
Validation prc-auc = 0.282541
Epoch 254
Loss = 1.0802e-04, PNorm = 89.2060, GNorm = 0.0025, lr_0 = 6.5491e-04
Validation prc-auc = 0.282478
Epoch 255
Validation prc-auc = 0.282478
Epoch 256
Validation prc-auc = 0.282416
Epoch 257
Validation prc-auc = 0.282416
Epoch 258
Validation prc-auc = 0.282355
Epoch 259
Loss = 8.0309e-05, PNorm = 89.2132, GNorm = 0.0017, lr_0 = 6.4931e-04
Validation prc-auc = 0.282355
Epoch 260
Validation prc-auc = 0.282355
Epoch 261
Validation prc-auc = 0.282948
Epoch 262
Validation prc-auc = 0.282948
Epoch 263
Validation prc-auc = 0.282948
Epoch 264
Loss = 1.2134e-04, PNorm = 89.2205, GNorm = 0.0028, lr_0 = 6.4376e-04
Validation prc-auc = 0.283067
Epoch 265
Validation prc-auc = 0.283067
Epoch 266
Validation prc-auc = 0.283067
Epoch 267
Validation prc-auc = 0.283067
Epoch 268
Validation prc-auc = 0.283067
Epoch 269
Loss = 1.1993e-04, PNorm = 89.2285, GNorm = 0.0048, lr_0 = 6.3826e-04
Validation prc-auc = 0.283122
Epoch 270
Validation prc-auc = 0.283122
Epoch 271
Validation prc-auc = 0.283183
Epoch 272
Validation prc-auc = 0.283183
Epoch 273
Validation prc-auc = 0.283183
Epoch 274
Loss = 5.0126e-05, PNorm = 89.2354, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.282589
Epoch 275
Validation prc-auc = 0.282635
Epoch 276
Validation prc-auc = 0.282635
Epoch 277
Validation prc-auc = 0.282635
Epoch 278
Validation prc-auc = 0.282589
Epoch 279
Loss = 5.5134e-05, PNorm = 89.2421, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation prc-auc = 0.282711
Epoch 280
Validation prc-auc = 0.282711
Epoch 281
Validation prc-auc = 0.282711
Epoch 282
Validation prc-auc = 0.282711
Epoch 283
Validation prc-auc = 0.282760
Epoch 284
Loss = 5.5362e-05, PNorm = 89.2500, GNorm = 0.0021, lr_0 = 6.2203e-04
Validation prc-auc = 0.282760
Epoch 285
Validation prc-auc = 0.282822
Epoch 286
Validation prc-auc = 0.282868
Epoch 287
Validation prc-auc = 0.282822
Epoch 288
Validation prc-auc = 0.283000
Epoch 289
Loss = 8.0032e-05, PNorm = 89.2569, GNorm = 0.0035, lr_0 = 6.1671e-04
Validation prc-auc = 0.283046
Epoch 290
Validation prc-auc = 0.282986
Epoch 291
Validation prc-auc = 0.283042
Epoch 292
Validation prc-auc = 0.283172
Epoch 293
Validation prc-auc = 0.283172
Epoch 294
Loss = 7.5541e-05, PNorm = 89.2602, GNorm = 0.0012, lr_0 = 6.1144e-04
Validation prc-auc = 0.283532
Epoch 295
Validation prc-auc = 0.283470
Epoch 296
Validation prc-auc = 0.283409
Epoch 297
Validation prc-auc = 0.283409
Epoch 298
Validation prc-auc = 0.283409
Epoch 299
Loss = 6.5809e-05, PNorm = 89.2681, GNorm = 0.0023, lr_0 = 6.0621e-04
Validation prc-auc = 0.283350
Model 0 best validation prc-auc = 0.490912 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.426883
Ensemble test prc-auc = 0.426883
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.450353
Epoch 1
Validation prc-auc = 0.443115
Epoch 2
Validation prc-auc = 0.453604
Epoch 3
Validation prc-auc = 0.464479
Epoch 4
Loss = 6.7479e-01, PNorm = 78.5519, GNorm = 9.3207, lr_0 = 6.5000e-04
Validation prc-auc = 0.438051
Epoch 5
Validation prc-auc = 0.437230
Epoch 6
Validation prc-auc = 0.488131
Epoch 7
Validation prc-auc = 0.433817
Epoch 8
Validation prc-auc = 0.490877
Epoch 9
Loss = 5.3238e-01, PNorm = 78.7874, GNorm = 4.2839, lr_0 = 9.9743e-04
Validation prc-auc = 0.451423
Epoch 10
Validation prc-auc = 0.447778
Epoch 11
Validation prc-auc = 0.250358
Epoch 12
Validation prc-auc = 0.251147
Epoch 13
Validation prc-auc = 0.293715
Epoch 14
Loss = 5.4735e-01, PNorm = 79.0682, GNorm = 4.1783, lr_0 = 9.8890e-04
Validation prc-auc = 0.430211
Epoch 15
Validation prc-auc = 0.442787
Epoch 16
Validation prc-auc = 0.435221
Epoch 17
Validation prc-auc = 0.442071
Epoch 18
Validation prc-auc = 0.437712
Epoch 19
Loss = 5.9960e-01, PNorm = 79.3356, GNorm = 5.0889, lr_0 = 9.8045e-04
Validation prc-auc = 0.405947
Epoch 20
Validation prc-auc = 0.415426
Epoch 21
Validation prc-auc = 0.398738
Epoch 22
Validation prc-auc = 0.191386
Epoch 23
Validation prc-auc = 0.332591
Epoch 24
Loss = 5.0185e-01, PNorm = 79.6336, GNorm = 1.1886, lr_0 = 9.7207e-04
Validation prc-auc = 0.352359
Epoch 25
Validation prc-auc = 0.346060
Epoch 26
Validation prc-auc = 0.354681
Epoch 27
Validation prc-auc = 0.374514
Epoch 28
Validation prc-auc = 0.359520
Epoch 29
Loss = 3.9279e-01, PNorm = 79.9565, GNorm = 3.0733, lr_0 = 9.6376e-04
Validation prc-auc = 0.394954
Epoch 30
Validation prc-auc = 0.378815
Epoch 31
Validation prc-auc = 0.363490
Epoch 32
Validation prc-auc = 0.284182
Epoch 33
Validation prc-auc = 0.335491
Epoch 34
Loss = 4.0500e-01, PNorm = 80.2851, GNorm = 7.8650, lr_0 = 9.5552e-04
Validation prc-auc = 0.349869
Epoch 35
Validation prc-auc = 0.369388
Epoch 36
Validation prc-auc = 0.391851
Epoch 37
Validation prc-auc = 0.375983
Epoch 38
Validation prc-auc = 0.399716
Epoch 39
Loss = 3.6006e-01, PNorm = 80.6638, GNorm = 2.7468, lr_0 = 9.4735e-04
Validation prc-auc = 0.382586
Epoch 40
Validation prc-auc = 0.383708
Epoch 41
Validation prc-auc = 0.379438
Epoch 42
Validation prc-auc = 0.344584
Epoch 43
Validation prc-auc = 0.356027
Epoch 44
Loss = 3.3708e-01, PNorm = 81.0584, GNorm = 1.1960, lr_0 = 9.3925e-04
Validation prc-auc = 0.356214
Epoch 45
Validation prc-auc = 0.333767
Epoch 46
Validation prc-auc = 0.313628
Epoch 47
Validation prc-auc = 0.336772
Epoch 48
Validation prc-auc = 0.330730
Epoch 49
Loss = 1.9644e-01, PNorm = 81.4938, GNorm = 3.2408, lr_0 = 9.3122e-04
Validation prc-auc = 0.354519
Epoch 50
Validation prc-auc = 0.351713
Epoch 51
Validation prc-auc = 0.354549
Epoch 52
Validation prc-auc = 0.319716
Epoch 53
Validation prc-auc = 0.306216
Epoch 54
Loss = 1.4992e-01, PNorm = 81.9167, GNorm = 0.7446, lr_0 = 9.2326e-04
Validation prc-auc = 0.327580
Epoch 55
Validation prc-auc = 0.325589
Epoch 56
Validation prc-auc = 0.352842
Epoch 57
Validation prc-auc = 0.354549
Epoch 58
Validation prc-auc = 0.348195
Epoch 59
Loss = 1.6586e-01, PNorm = 82.3132, GNorm = 1.7308, lr_0 = 9.1537e-04
Validation prc-auc = 0.305380
Epoch 60
Validation prc-auc = 0.332003
Epoch 61
Validation prc-auc = 0.352183
Epoch 62
Validation prc-auc = 0.269207
Epoch 63
Validation prc-auc = 0.257248
Epoch 64
Loss = 1.8356e-01, PNorm = 82.7234, GNorm = 6.7183, lr_0 = 9.0754e-04
Validation prc-auc = 0.353868
Epoch 65
Validation prc-auc = 0.302649
Epoch 66
Validation prc-auc = 0.318274
Epoch 67
Validation prc-auc = 0.329895
Epoch 68
Validation prc-auc = 0.313174
Epoch 69
Loss = 1.1830e-01, PNorm = 83.1548, GNorm = 0.8571, lr_0 = 8.9978e-04
Validation prc-auc = 0.253649
Epoch 70
Validation prc-auc = 0.259100
Epoch 71
Validation prc-auc = 0.328664
Epoch 72
Validation prc-auc = 0.267724
Epoch 73
Validation prc-auc = 0.286050
Epoch 74
Loss = 6.8920e-02, PNorm = 83.5520, GNorm = 1.7007, lr_0 = 8.9209e-04
Validation prc-auc = 0.352845
Epoch 75
Validation prc-auc = 0.351592
Epoch 76
Validation prc-auc = 0.323956
Epoch 77
Validation prc-auc = 0.292979
Epoch 78
Validation prc-auc = 0.285621
Epoch 79
Loss = 2.5831e-02, PNorm = 83.8893, GNorm = 0.1329, lr_0 = 8.8447e-04
Validation prc-auc = 0.292541
Epoch 80
Validation prc-auc = 0.287925
Epoch 81
Validation prc-auc = 0.297752
Epoch 82
Validation prc-auc = 0.307407
Epoch 83
Validation prc-auc = 0.260515
Epoch 84
Loss = 2.5320e-02, PNorm = 84.1755, GNorm = 0.3427, lr_0 = 8.7691e-04
Validation prc-auc = 0.189323
Epoch 85
Validation prc-auc = 0.225363
Epoch 86
Validation prc-auc = 0.286895
Epoch 87
Validation prc-auc = 0.292944
Epoch 88
Validation prc-auc = 0.292704
Epoch 89
Loss = 2.5646e-01, PNorm = 84.4516, GNorm = 2.0814, lr_0 = 8.6941e-04
Validation prc-auc = 0.360888
Epoch 90
Validation prc-auc = 0.360951
Epoch 91
Validation prc-auc = 0.348303
Epoch 92
Validation prc-auc = 0.320626
Epoch 93
Validation prc-auc = 0.346472
Epoch 94
Loss = 6.0640e-02, PNorm = 84.7099, GNorm = 1.4590, lr_0 = 8.6198e-04
Validation prc-auc = 0.360081
Epoch 95
Validation prc-auc = 0.356844
Epoch 96
Validation prc-auc = 0.342179
Epoch 97
Validation prc-auc = 0.285905
Epoch 98
Validation prc-auc = 0.196533
Epoch 99
Loss = 5.2903e-02, PNorm = 84.9901, GNorm = 0.4358, lr_0 = 8.5461e-04
Validation prc-auc = 0.229233
Epoch 100
Validation prc-auc = 0.249626
Epoch 101
Validation prc-auc = 0.258652
Epoch 102
Validation prc-auc = 0.271943
Epoch 103
Validation prc-auc = 0.290447
Epoch 104
Loss = 2.2823e-02, PNorm = 85.2376, GNorm = 0.5505, lr_0 = 8.4730e-04
Validation prc-auc = 0.321223
Epoch 105
Validation prc-auc = 0.308526
Epoch 106
Validation prc-auc = 0.312397
Epoch 107
Validation prc-auc = 0.292327
Epoch 108
Validation prc-auc = 0.325420
Epoch 109
Loss = 2.6738e-02, PNorm = 85.4408, GNorm = 0.6383, lr_0 = 8.4006e-04
Validation prc-auc = 0.325156
Epoch 110
Validation prc-auc = 0.317045
Epoch 111
Validation prc-auc = 0.318840
Epoch 112
Validation prc-auc = 0.311052
Epoch 113
Validation prc-auc = 0.269121
Epoch 114
Loss = 1.9127e-02, PNorm = 85.6330, GNorm = 1.2177, lr_0 = 8.3288e-04
Validation prc-auc = 0.266165
Epoch 115
Validation prc-auc = 0.249533
Epoch 116
Validation prc-auc = 0.220490
Epoch 117
Validation prc-auc = 0.217333
Epoch 118
Validation prc-auc = 0.242705
Epoch 119
Loss = 2.5329e-03, PNorm = 85.7843, GNorm = 0.0279, lr_0 = 8.2576e-04
Validation prc-auc = 0.251771
Epoch 120
Validation prc-auc = 0.254955
Epoch 121
Validation prc-auc = 0.253266
Epoch 122
Validation prc-auc = 0.236312
Epoch 123
Validation prc-auc = 0.235802
Epoch 124
Loss = 4.2049e-03, PNorm = 85.9025, GNorm = 0.0175, lr_0 = 8.1870e-04
Validation prc-auc = 0.245836
Epoch 125
Validation prc-auc = 0.262948
Epoch 126
Validation prc-auc = 0.274168
Epoch 127
Validation prc-auc = 0.286154
Epoch 128
Validation prc-auc = 0.286708
Epoch 129
Loss = 6.6386e-03, PNorm = 86.0234, GNorm = 0.1598, lr_0 = 8.1170e-04
Validation prc-auc = 0.286670
Epoch 130
Validation prc-auc = 0.280666
Epoch 131
Validation prc-auc = 0.276780
Epoch 132
Validation prc-auc = 0.271997
Epoch 133
Validation prc-auc = 0.271172
Epoch 134
Loss = 2.2648e-03, PNorm = 86.1266, GNorm = 0.0519, lr_0 = 8.0476e-04
Validation prc-auc = 0.273403
Epoch 135
Validation prc-auc = 0.284740
Epoch 136
Validation prc-auc = 0.281803
Epoch 137
Validation prc-auc = 0.269278
Epoch 138
Validation prc-auc = 0.268204
Epoch 139
Loss = 5.9064e-03, PNorm = 86.1971, GNorm = 0.1472, lr_0 = 7.9788e-04
Validation prc-auc = 0.248496
Epoch 140
Validation prc-auc = 0.224663
Epoch 141
Validation prc-auc = 0.237987
Epoch 142
Validation prc-auc = 0.251706
Epoch 143
Validation prc-auc = 0.263135
Epoch 144
Loss = 5.0731e-03, PNorm = 86.2741, GNorm = 0.1962, lr_0 = 7.9106e-04
Validation prc-auc = 0.269497
Epoch 145
Validation prc-auc = 0.288364
Epoch 146
Validation prc-auc = 0.299853
Epoch 147
Validation prc-auc = 0.300752
Epoch 148
Validation prc-auc = 0.307451
Epoch 149
Loss = 4.7443e-03, PNorm = 86.3435, GNorm = 0.1533, lr_0 = 7.8430e-04
Validation prc-auc = 0.308677
Epoch 150
Validation prc-auc = 0.307553
Epoch 151
Validation prc-auc = 0.303300
Epoch 152
Validation prc-auc = 0.301494
Epoch 153
Validation prc-auc = 0.290292
Epoch 154
Loss = 4.1294e-04, PNorm = 86.4086, GNorm = 0.0088, lr_0 = 7.7759e-04
Validation prc-auc = 0.287722
Epoch 155
Validation prc-auc = 0.276216
Epoch 156
Validation prc-auc = 0.275685
Epoch 157
Validation prc-auc = 0.275287
Epoch 158
Validation prc-auc = 0.274977
Epoch 159
Loss = 5.0478e-04, PNorm = 86.4433, GNorm = 0.0198, lr_0 = 7.7095e-04
Validation prc-auc = 0.285617
Epoch 160
Validation prc-auc = 0.291283
Epoch 161
Validation prc-auc = 0.294036
Epoch 162
Validation prc-auc = 0.295108
Epoch 163
Validation prc-auc = 0.294899
Epoch 164
Loss = 1.1425e-03, PNorm = 86.4712, GNorm = 0.0720, lr_0 = 7.6436e-04
Validation prc-auc = 0.303949
Epoch 165
Validation prc-auc = 0.304318
Epoch 166
Validation prc-auc = 0.305851
Epoch 167
Validation prc-auc = 0.308465
Epoch 168
Validation prc-auc = 0.309083
Epoch 169
Loss = 2.4120e-03, PNorm = 86.5002, GNorm = 0.1663, lr_0 = 7.5782e-04
Validation prc-auc = 0.304228
Epoch 170
Validation prc-auc = 0.302654
Epoch 171
Validation prc-auc = 0.300803
Epoch 172
Validation prc-auc = 0.288076
Epoch 173
Validation prc-auc = 0.266056
Epoch 174
Loss = 5.1762e-03, PNorm = 86.5392, GNorm = 0.3394, lr_0 = 7.5134e-04
Validation prc-auc = 0.253052
Epoch 175
Validation prc-auc = 0.249086
Epoch 176
Validation prc-auc = 0.240232
Epoch 177
Validation prc-auc = 0.236760
Epoch 178
Validation prc-auc = 0.245920
Epoch 179
Loss = 4.4986e-04, PNorm = 86.6111, GNorm = 0.0509, lr_0 = 7.4492e-04
Validation prc-auc = 0.255787
Epoch 180
Validation prc-auc = 0.253410
Epoch 181
Validation prc-auc = 0.247341
Epoch 182
Validation prc-auc = 0.251789
Epoch 183
Validation prc-auc = 0.252916
Epoch 184
Loss = 3.4637e-04, PNorm = 86.6513, GNorm = 0.0046, lr_0 = 7.3855e-04
Validation prc-auc = 0.252702
Epoch 185
Validation prc-auc = 0.255836
Epoch 186
Validation prc-auc = 0.261791
Epoch 187
Validation prc-auc = 0.261721
Epoch 188
Validation prc-auc = 0.261903
Epoch 189
Loss = 4.1657e-04, PNorm = 86.6766, GNorm = 0.0157, lr_0 = 7.3224e-04
Validation prc-auc = 0.261880
Epoch 190
Validation prc-auc = 0.261832
Epoch 191
Validation prc-auc = 0.261858
Epoch 192
Validation prc-auc = 0.263395
Epoch 193
Validation prc-auc = 0.263395
Epoch 194
Loss = 3.9351e-04, PNorm = 86.6943, GNorm = 0.0241, lr_0 = 7.2598e-04
Validation prc-auc = 0.263539
Epoch 195
Validation prc-auc = 0.263459
Epoch 196
Validation prc-auc = 0.257308
Epoch 197
Validation prc-auc = 0.257389
Epoch 198
Validation prc-auc = 0.257389
Epoch 199
Loss = 1.3199e-04, PNorm = 86.7039, GNorm = 0.0050, lr_0 = 7.1977e-04
Validation prc-auc = 0.255762
Epoch 200
Validation prc-auc = 0.255898
Epoch 201
Validation prc-auc = 0.255898
Epoch 202
Validation prc-auc = 0.256025
Epoch 203
Validation prc-auc = 0.256097
Epoch 204
Loss = 1.1349e-04, PNorm = 86.7099, GNorm = 0.0032, lr_0 = 7.1362e-04
Validation prc-auc = 0.256097
Epoch 205
Validation prc-auc = 0.256099
Epoch 206
Validation prc-auc = 0.256112
Epoch 207
Validation prc-auc = 0.256198
Epoch 208
Validation prc-auc = 0.249690
Epoch 209
Loss = 8.2639e-05, PNorm = 86.7144, GNorm = 0.0027, lr_0 = 7.0752e-04
Validation prc-auc = 0.249690
Epoch 210
Validation prc-auc = 0.249690
Epoch 211
Validation prc-auc = 0.249690
Epoch 212
Validation prc-auc = 0.249690
Epoch 213
Validation prc-auc = 0.249779
Epoch 214
Loss = 8.4569e-05, PNorm = 86.7185, GNorm = 0.0030, lr_0 = 7.0147e-04
Validation prc-auc = 0.249779
Epoch 215
Validation prc-auc = 0.249762
Epoch 216
Validation prc-auc = 0.249890
Epoch 217
Validation prc-auc = 0.249953
Epoch 218
Validation prc-auc = 0.249953
Epoch 219
Loss = 8.3627e-05, PNorm = 86.7223, GNorm = 0.0024, lr_0 = 6.9548e-04
Validation prc-auc = 0.256533
Epoch 220
Validation prc-auc = 0.256463
Epoch 221
Validation prc-auc = 0.256555
Epoch 222
Validation prc-auc = 0.256555
Epoch 223
Validation prc-auc = 0.257064
Epoch 224
Loss = 7.8145e-05, PNorm = 86.7260, GNorm = 0.0025, lr_0 = 6.8953e-04
Validation prc-auc = 0.257064
Epoch 225
Validation prc-auc = 0.257129
Epoch 226
Validation prc-auc = 0.257129
Epoch 227
Validation prc-auc = 0.257129
Epoch 228
Validation prc-auc = 0.257129
Epoch 229
Loss = 1.1600e-04, PNorm = 86.7296, GNorm = 0.0026, lr_0 = 6.8364e-04
Validation prc-auc = 0.257129
Epoch 230
Validation prc-auc = 0.257129
Epoch 231
Validation prc-auc = 0.257129
Epoch 232
Validation prc-auc = 0.257129
Epoch 233
Validation prc-auc = 0.257129
Epoch 234
Loss = 1.8573e-04, PNorm = 86.7332, GNorm = 0.0031, lr_0 = 6.7779e-04
Validation prc-auc = 0.257065
Epoch 235
Validation prc-auc = 0.257065
Epoch 236
Validation prc-auc = 0.257199
Epoch 237
Validation prc-auc = 0.257277
Epoch 238
Validation prc-auc = 0.257344
Epoch 239
Loss = 9.2596e-05, PNorm = 86.7372, GNorm = 0.0019, lr_0 = 6.7200e-04
Validation prc-auc = 0.257278
Epoch 240
Validation prc-auc = 0.257255
Epoch 241
Validation prc-auc = 0.257255
Epoch 242
Validation prc-auc = 0.257255
Epoch 243
Validation prc-auc = 0.257255
Epoch 244
Loss = 6.1965e-05, PNorm = 86.7413, GNorm = 0.0012, lr_0 = 6.6625e-04
Validation prc-auc = 0.257184
Epoch 245
Validation prc-auc = 0.257184
Epoch 246
Validation prc-auc = 0.257115
Epoch 247
Validation prc-auc = 0.257206
Epoch 248
Validation prc-auc = 0.257206
Epoch 249
Loss = 6.6423e-05, PNorm = 86.7452, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.257139
Epoch 250
Validation prc-auc = 0.257139
Epoch 251
Validation prc-auc = 0.257097
Epoch 252
Validation prc-auc = 0.257097
Epoch 253
Validation prc-auc = 0.257097
Epoch 254
Loss = 4.9712e-05, PNorm = 86.7488, GNorm = 0.0019, lr_0 = 6.5491e-04
Validation prc-auc = 0.257082
Epoch 255
Validation prc-auc = 0.257006
Epoch 256
Validation prc-auc = 0.257082
Epoch 257
Validation prc-auc = 0.257071
Epoch 258
Validation prc-auc = 0.257147
Epoch 259
Loss = 5.1863e-05, PNorm = 86.7524, GNorm = 0.0013, lr_0 = 6.4931e-04
Validation prc-auc = 0.256987
Epoch 260
Validation prc-auc = 0.256963
Epoch 261
Validation prc-auc = 0.256963
Epoch 262
Validation prc-auc = 0.256963
Epoch 263
Validation prc-auc = 0.257030
Epoch 264
Loss = 1.9087e-04, PNorm = 86.7558, GNorm = 0.0013, lr_0 = 6.4376e-04
Validation prc-auc = 0.257163
Epoch 265
Validation prc-auc = 0.257311
Epoch 266
Validation prc-auc = 0.257779
Epoch 267
Validation prc-auc = 0.257779
Epoch 268
Validation prc-auc = 0.257846
Epoch 269
Loss = 7.9699e-05, PNorm = 86.7595, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.263997
Epoch 270
Validation prc-auc = 0.263997
Epoch 271
Validation prc-auc = 0.264088
Epoch 272
Validation prc-auc = 0.264019
Epoch 273
Validation prc-auc = 0.264019
Epoch 274
Loss = 4.5145e-05, PNorm = 86.7629, GNorm = 0.0011, lr_0 = 6.3280e-04
Validation prc-auc = 0.264019
Epoch 275
Validation prc-auc = 0.257869
Epoch 276
Validation prc-auc = 0.264510
Epoch 277
Validation prc-auc = 0.264510
Epoch 278
Validation prc-auc = 0.264443
Epoch 279
Loss = 3.9168e-05, PNorm = 86.7663, GNorm = 0.0010, lr_0 = 6.2739e-04
Validation prc-auc = 0.264443
Epoch 280
Validation prc-auc = 0.264443
Epoch 281
Validation prc-auc = 0.264443
Epoch 282
Validation prc-auc = 0.264537
Epoch 283
Validation prc-auc = 0.264537
Epoch 284
Loss = 1.1023e-04, PNorm = 86.7693, GNorm = 0.0100, lr_0 = 6.2203e-04
Validation prc-auc = 0.264537
Epoch 285
Validation prc-auc = 0.264523
Epoch 286
Validation prc-auc = 0.264590
Epoch 287
Validation prc-auc = 0.264659
Epoch 288
Validation prc-auc = 0.264728
Epoch 289
Loss = 3.2019e-05, PNorm = 86.7725, GNorm = 0.0013, lr_0 = 6.1671e-04
Validation prc-auc = 0.264728
Epoch 290
Validation prc-auc = 0.264728
Epoch 291
Validation prc-auc = 0.264728
Epoch 292
Validation prc-auc = 0.264728
Epoch 293
Validation prc-auc = 0.264718
Epoch 294
Loss = 3.1254e-05, PNorm = 86.7757, GNorm = 0.0010, lr_0 = 6.1144e-04
Validation prc-auc = 0.264718
Epoch 295
Validation prc-auc = 0.264718
Epoch 296
Validation prc-auc = 0.264609
Epoch 297
Validation prc-auc = 0.264609
Epoch 298
Validation prc-auc = 0.264978
Epoch 299
Loss = 3.0260e-05, PNorm = 86.7789, GNorm = 0.0009, lr_0 = 6.0621e-04
Validation prc-auc = 0.264978
Model 0 best validation prc-auc = 0.490877 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387804
Ensemble test prc-auc = 0.387804
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.421469
Epoch 1
Validation prc-auc = 0.423979
Epoch 2
Validation prc-auc = 0.439751
Epoch 3
Validation prc-auc = 0.465896
Epoch 4
Loss = 7.3687e-01, PNorm = 78.5527, GNorm = 8.0153, lr_0 = 6.5000e-04
Validation prc-auc = 0.443333
Epoch 5
Validation prc-auc = 0.463636
Epoch 6
Validation prc-auc = 0.433615
Epoch 7
Validation prc-auc = 0.319327
Epoch 8
Validation prc-auc = 0.433522
Epoch 9
Loss = 5.4549e-01, PNorm = 78.7983, GNorm = 1.2378, lr_0 = 9.9743e-04
Validation prc-auc = 0.439056
Epoch 10
Validation prc-auc = 0.451785
Epoch 11
Validation prc-auc = 0.362315
Epoch 12
Validation prc-auc = 0.331306
Epoch 13
Validation prc-auc = 0.316753
Epoch 14
Loss = 5.2377e-01, PNorm = 79.1024, GNorm = 0.8953, lr_0 = 9.8890e-04
Validation prc-auc = 0.320442
Epoch 15
Validation prc-auc = 0.295246
Epoch 16
Validation prc-auc = 0.325186
Epoch 17
Validation prc-auc = 0.353658
Epoch 18
Validation prc-auc = 0.353847
Epoch 19
Loss = 4.4552e-01, PNorm = 79.4384, GNorm = 1.8933, lr_0 = 9.8045e-04
Validation prc-auc = 0.304821
Epoch 20
Validation prc-auc = 0.304510
Epoch 21
Validation prc-auc = 0.334042
Epoch 22
Validation prc-auc = 0.388249
Epoch 23
Validation prc-auc = 0.385059
Epoch 24
Loss = 3.6079e-01, PNorm = 79.7877, GNorm = 2.4063, lr_0 = 9.7207e-04
Validation prc-auc = 0.456953
Epoch 25
Validation prc-auc = 0.445516
Epoch 26
Validation prc-auc = 0.298666
Epoch 27
Validation prc-auc = 0.266809
Epoch 28
Validation prc-auc = 0.268973
Epoch 29
Loss = 3.4658e-01, PNorm = 80.1289, GNorm = 0.8745, lr_0 = 9.6376e-04
Validation prc-auc = 0.318488
Epoch 30
Validation prc-auc = 0.326917
Epoch 31
Validation prc-auc = 0.329532
Epoch 32
Validation prc-auc = 0.325767
Epoch 33
Validation prc-auc = 0.166782
Epoch 34
Loss = 2.9142e-01, PNorm = 80.4997, GNorm = 1.4784, lr_0 = 9.5552e-04
Validation prc-auc = 0.263719
Epoch 35
Validation prc-auc = 0.281547
Epoch 36
Validation prc-auc = 0.323984
Epoch 37
Validation prc-auc = 0.341547
Epoch 38
Validation prc-auc = 0.352572
Epoch 39
Loss = 3.0176e-01, PNorm = 80.9185, GNorm = 1.8169, lr_0 = 9.4735e-04
Validation prc-auc = 0.345094
Epoch 40
Validation prc-auc = 0.307132
Epoch 41
Validation prc-auc = 0.325801
Epoch 42
Validation prc-auc = 0.341158
Epoch 43
Validation prc-auc = 0.332618
Epoch 44
Loss = 1.5395e-01, PNorm = 81.3193, GNorm = 1.4727, lr_0 = 9.3925e-04
Validation prc-auc = 0.346512
Epoch 45
Validation prc-auc = 0.314487
Epoch 46
Validation prc-auc = 0.321420
Epoch 47
Validation prc-auc = 0.331676
Epoch 48
Validation prc-auc = 0.325380
Epoch 49
Loss = 8.4553e-02, PNorm = 81.7248, GNorm = 0.4701, lr_0 = 9.3122e-04
Validation prc-auc = 0.336953
Epoch 50
Validation prc-auc = 0.317782
Epoch 51
Validation prc-auc = 0.334362
Epoch 52
Validation prc-auc = 0.344895
Epoch 53
Validation prc-auc = 0.338960
Epoch 54
Loss = 1.1898e-01, PNorm = 82.1275, GNorm = 1.7424, lr_0 = 9.2326e-04
Validation prc-auc = 0.312097
Epoch 55
Validation prc-auc = 0.224829
Epoch 56
Validation prc-auc = 0.211702
Epoch 57
Validation prc-auc = 0.228625
Epoch 58
Validation prc-auc = 0.249902
Epoch 59
Loss = 1.3664e-01, PNorm = 82.5256, GNorm = 0.8003, lr_0 = 9.1537e-04
Validation prc-auc = 0.315017
Epoch 60
Validation prc-auc = 0.171745
Epoch 61
Validation prc-auc = 0.254731
Epoch 62
Validation prc-auc = 0.270841
Epoch 63
Validation prc-auc = 0.216812
Epoch 64
Loss = 2.4207e-01, PNorm = 82.9790, GNorm = 3.0624, lr_0 = 9.0754e-04
Validation prc-auc = 0.303712
Epoch 65
Validation prc-auc = 0.300287
Epoch 66
Validation prc-auc = 0.227536
Epoch 67
Validation prc-auc = 0.234641
Epoch 68
Validation prc-auc = 0.293496
Epoch 69
Loss = 4.8540e-02, PNorm = 83.3779, GNorm = 0.2424, lr_0 = 8.9978e-04
Validation prc-auc = 0.277111
Epoch 70
Validation prc-auc = 0.254592
Epoch 71
Validation prc-auc = 0.258545
Epoch 72
Validation prc-auc = 0.237237
Epoch 73
Validation prc-auc = 0.219126
Epoch 74
Loss = 1.3329e-01, PNorm = 83.6973, GNorm = 2.4291, lr_0 = 8.9209e-04
Validation prc-auc = 0.237532
Epoch 75
Validation prc-auc = 0.266272
Epoch 76
Validation prc-auc = 0.282868
Epoch 77
Validation prc-auc = 0.245745
Epoch 78
Validation prc-auc = 0.281975
Epoch 79
Loss = 3.0196e-02, PNorm = 83.9751, GNorm = 0.2708, lr_0 = 8.8447e-04
Validation prc-auc = 0.328357
Epoch 80
Validation prc-auc = 0.321928
Epoch 81
Validation prc-auc = 0.235448
Epoch 82
Validation prc-auc = 0.195459
Epoch 83
Validation prc-auc = 0.192653
Epoch 84
Loss = 2.9629e-02, PNorm = 84.2209, GNorm = 0.1373, lr_0 = 8.7691e-04
Validation prc-auc = 0.239309
Epoch 85
Validation prc-auc = 0.304257
Epoch 86
Validation prc-auc = 0.333307
Epoch 87
Validation prc-auc = 0.289372
Epoch 88
Validation prc-auc = 0.236526
Epoch 89
Loss = 2.6510e-02, PNorm = 84.4304, GNorm = 0.0461, lr_0 = 8.6941e-04
Validation prc-auc = 0.198907
Epoch 90
Validation prc-auc = 0.251381
Epoch 91
Validation prc-auc = 0.278850
Epoch 92
Validation prc-auc = 0.279741
Epoch 93
Validation prc-auc = 0.248908
Epoch 94
Loss = 4.4098e-02, PNorm = 84.6117, GNorm = 0.1094, lr_0 = 8.6198e-04
Validation prc-auc = 0.217561
Epoch 95
Validation prc-auc = 0.220587
Epoch 96
Validation prc-auc = 0.230401
Epoch 97
Validation prc-auc = 0.225088
Epoch 98
Validation prc-auc = 0.193584
Epoch 99
Loss = 1.4977e-02, PNorm = 84.7991, GNorm = 0.3501, lr_0 = 8.5461e-04
Validation prc-auc = 0.176868
Epoch 100
Validation prc-auc = 0.170880
Epoch 101
Validation prc-auc = 0.172012
Epoch 102
Validation prc-auc = 0.185316
Epoch 103
Validation prc-auc = 0.199213
Epoch 104
Loss = 5.0745e-03, PNorm = 84.9753, GNorm = 0.2658, lr_0 = 8.4730e-04
Validation prc-auc = 0.211431
Epoch 105
Validation prc-auc = 0.217693
Epoch 106
Validation prc-auc = 0.230831
Epoch 107
Validation prc-auc = 0.240978
Epoch 108
Validation prc-auc = 0.263327
Epoch 109
Loss = 1.2869e-02, PNorm = 85.1049, GNorm = 1.0465, lr_0 = 8.4006e-04
Validation prc-auc = 0.267876
Epoch 110
Validation prc-auc = 0.234446
Epoch 111
Validation prc-auc = 0.221977
Epoch 112
Validation prc-auc = 0.209011
Epoch 113
Validation prc-auc = 0.207965
Epoch 114
Loss = 2.2331e-03, PNorm = 85.1923, GNorm = 0.1035, lr_0 = 8.3288e-04
Validation prc-auc = 0.216497
Epoch 115
Validation prc-auc = 0.226325
Epoch 116
Validation prc-auc = 0.227724
Epoch 117
Validation prc-auc = 0.235186
Epoch 118
Validation prc-auc = 0.237335
Epoch 119
Loss = 2.6992e-02, PNorm = 85.2644, GNorm = 0.3377, lr_0 = 8.2576e-04
Validation prc-auc = 0.266233
Epoch 120
Validation prc-auc = 0.261343
Epoch 121
Validation prc-auc = 0.233877
Epoch 122
Validation prc-auc = 0.229014
Epoch 123
Validation prc-auc = 0.238096
Epoch 124
Loss = 6.5969e-02, PNorm = 85.3855, GNorm = 1.1910, lr_0 = 8.1870e-04
Validation prc-auc = 0.269577
Epoch 125
Validation prc-auc = 0.276810
Epoch 126
Validation prc-auc = 0.269922
Epoch 127
Validation prc-auc = 0.243705
Epoch 128
Validation prc-auc = 0.224063
Epoch 129
Loss = 1.1894e-02, PNorm = 85.5518, GNorm = 0.3324, lr_0 = 8.1170e-04
Validation prc-auc = 0.202577
Epoch 130
Validation prc-auc = 0.213873
Epoch 131
Validation prc-auc = 0.218412
Epoch 132
Validation prc-auc = 0.245407
Epoch 133
Validation prc-auc = 0.270409
Epoch 134
Loss = 1.1480e-02, PNorm = 85.7105, GNorm = 0.5125, lr_0 = 8.0476e-04
Validation prc-auc = 0.270824
Epoch 135
Validation prc-auc = 0.270837
Epoch 136
Validation prc-auc = 0.268830
Epoch 137
Validation prc-auc = 0.260741
Epoch 138
Validation prc-auc = 0.263618
Epoch 139
Loss = 1.1048e-02, PNorm = 85.8041, GNorm = 0.5878, lr_0 = 7.9788e-04
Validation prc-auc = 0.224960
Epoch 140
Validation prc-auc = 0.222092
Epoch 141
Validation prc-auc = 0.220771
Epoch 142
Validation prc-auc = 0.220701
Epoch 143
Validation prc-auc = 0.214179
Epoch 144
Loss = 1.4845e-03, PNorm = 85.9090, GNorm = 0.1000, lr_0 = 7.9106e-04
Validation prc-auc = 0.215867
Epoch 145
Validation prc-auc = 0.214265
Epoch 146
Validation prc-auc = 0.216375
Epoch 147
Validation prc-auc = 0.213805
Epoch 148
Validation prc-auc = 0.214617
Epoch 149
Loss = 1.1243e-03, PNorm = 85.9787, GNorm = 0.0499, lr_0 = 7.8430e-04
Validation prc-auc = 0.212677
Epoch 150
Validation prc-auc = 0.217421
Epoch 151
Validation prc-auc = 0.225611
Epoch 152
Validation prc-auc = 0.226140
Epoch 153
Validation prc-auc = 0.226447
Epoch 154
Loss = 6.7331e-04, PNorm = 86.0258, GNorm = 0.0165, lr_0 = 7.7759e-04
Validation prc-auc = 0.225323
Epoch 155
Validation prc-auc = 0.228061
Epoch 156
Validation prc-auc = 0.228166
Epoch 157
Validation prc-auc = 0.231046
Epoch 158
Validation prc-auc = 0.230086
Epoch 159
Loss = 5.1876e-04, PNorm = 86.0613, GNorm = 0.0073, lr_0 = 7.7095e-04
Validation prc-auc = 0.230086
Epoch 160
Validation prc-auc = 0.231706
Epoch 161
Validation prc-auc = 0.230949
Epoch 162
Validation prc-auc = 0.226674
Epoch 163
Validation prc-auc = 0.224061
Epoch 164
Loss = 5.7099e-04, PNorm = 86.0859, GNorm = 0.0521, lr_0 = 7.6436e-04
Validation prc-auc = 0.229333
Epoch 165
Validation prc-auc = 0.235687
Epoch 166
Validation prc-auc = 0.235814
Epoch 167
Validation prc-auc = 0.236074
Epoch 168
Validation prc-auc = 0.235992
Epoch 169
Loss = 5.0852e-04, PNorm = 86.1070, GNorm = 0.0154, lr_0 = 7.5782e-04
Validation prc-auc = 0.235922
Epoch 170
Validation prc-auc = 0.232089
Epoch 171
Validation prc-auc = 0.231757
Epoch 172
Validation prc-auc = 0.231473
Epoch 173
Validation prc-auc = 0.231533
Epoch 174
Loss = 5.2526e-04, PNorm = 86.1244, GNorm = 0.0538, lr_0 = 7.5134e-04
Validation prc-auc = 0.231316
Epoch 175
Validation prc-auc = 0.231248
Epoch 176
Validation prc-auc = 0.231120
Epoch 177
Validation prc-auc = 0.230419
Epoch 178
Validation prc-auc = 0.230486
Epoch 179
Loss = 2.8719e-04, PNorm = 86.1390, GNorm = 0.0099, lr_0 = 7.4492e-04
Validation prc-auc = 0.230486
Epoch 180
Validation prc-auc = 0.230486
Epoch 181
Validation prc-auc = 0.230697
Epoch 182
Validation prc-auc = 0.230764
Epoch 183
Validation prc-auc = 0.230314
Epoch 184
Loss = 1.7340e-04, PNorm = 86.1509, GNorm = 0.0039, lr_0 = 7.3855e-04
Validation prc-auc = 0.230314
Epoch 185
Validation prc-auc = 0.234203
Epoch 186
Validation prc-auc = 0.234203
Epoch 187
Validation prc-auc = 0.234422
Epoch 188
Validation prc-auc = 0.234355
Epoch 189
Loss = 1.6721e-04, PNorm = 86.1599, GNorm = 0.0051, lr_0 = 7.3224e-04
Validation prc-auc = 0.234355
Epoch 190
Validation prc-auc = 0.234288
Epoch 191
Validation prc-auc = 0.238891
Epoch 192
Validation prc-auc = 0.238958
Epoch 193
Validation prc-auc = 0.238749
Epoch 194
Loss = 2.0894e-04, PNorm = 86.1678, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.238749
Epoch 195
Validation prc-auc = 0.238815
Epoch 196
Validation prc-auc = 0.238585
Epoch 197
Validation prc-auc = 0.238585
Epoch 198
Validation prc-auc = 0.238706
Epoch 199
Loss = 7.1296e-05, PNorm = 86.1752, GNorm = 0.0019, lr_0 = 7.1977e-04
Validation prc-auc = 0.240320
Epoch 200
Validation prc-auc = 0.240320
Epoch 201
Validation prc-auc = 0.240320
Epoch 202
Validation prc-auc = 0.240320
Epoch 203
Validation prc-auc = 0.240179
Epoch 204
Loss = 2.1907e-04, PNorm = 86.1820, GNorm = 0.0056, lr_0 = 7.1362e-04
Validation prc-auc = 0.239934
Epoch 205
Validation prc-auc = 0.240002
Epoch 206
Validation prc-auc = 0.239845
Epoch 207
Validation prc-auc = 0.239845
Epoch 208
Validation prc-auc = 0.239710
Epoch 209
Loss = 5.3842e-05, PNorm = 86.1887, GNorm = 0.0024, lr_0 = 7.0752e-04
Validation prc-auc = 0.239772
Epoch 210
Validation prc-auc = 0.239772
Epoch 211
Validation prc-auc = 0.239898
Epoch 212
Validation prc-auc = 0.239898
Epoch 213
Validation prc-auc = 0.239965
Epoch 214
Loss = 1.1752e-04, PNorm = 86.1954, GNorm = 0.0012, lr_0 = 7.0147e-04
Validation prc-auc = 0.240030
Epoch 215
Validation prc-auc = 0.240170
Epoch 216
Validation prc-auc = 0.240849
Epoch 217
Validation prc-auc = 0.240921
Epoch 218
Validation prc-auc = 0.241056
Epoch 219
Loss = 8.7771e-05, PNorm = 86.2030, GNorm = 0.0032, lr_0 = 6.9548e-04
Validation prc-auc = 0.244760
Epoch 220
Validation prc-auc = 0.244760
Epoch 221
Validation prc-auc = 0.244690
Epoch 222
Validation prc-auc = 0.244626
Epoch 223
Validation prc-auc = 0.244837
Epoch 224
Loss = 7.8586e-05, PNorm = 86.2105, GNorm = 0.0031, lr_0 = 6.8953e-04
Validation prc-auc = 0.244524
Epoch 225
Validation prc-auc = 0.244362
Epoch 226
Validation prc-auc = 0.249124
Epoch 227
Validation prc-auc = 0.249188
Epoch 228
Validation prc-auc = 0.249326
Epoch 229
Loss = 5.3647e-05, PNorm = 86.2180, GNorm = 0.0025, lr_0 = 6.8364e-04
Validation prc-auc = 0.249394
Epoch 230
Validation prc-auc = 0.249394
Epoch 231
Validation prc-auc = 0.249323
Epoch 232
Validation prc-auc = 0.249323
Epoch 233
Validation prc-auc = 0.249774
Epoch 234
Loss = 4.6062e-05, PNorm = 86.2253, GNorm = 0.0011, lr_0 = 6.7779e-04
Validation prc-auc = 0.249422
Epoch 235
Validation prc-auc = 0.249270
Epoch 236
Validation prc-auc = 0.249270
Epoch 237
Validation prc-auc = 0.249123
Epoch 238
Validation prc-auc = 0.249323
Epoch 239
Loss = 4.2064e-05, PNorm = 86.2315, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.249323
Epoch 240
Validation prc-auc = 0.249186
Epoch 241
Validation prc-auc = 0.248821
Epoch 242
Validation prc-auc = 0.249550
Epoch 243
Validation prc-auc = 0.249271
Epoch 244
Loss = 5.0311e-05, PNorm = 86.2382, GNorm = 0.0018, lr_0 = 6.6625e-04
Validation prc-auc = 0.249340
Epoch 245
Validation prc-auc = 0.249273
Epoch 246
Validation prc-auc = 0.249273
Epoch 247
Validation prc-auc = 0.249343
Epoch 248
Validation prc-auc = 0.249351
Epoch 249
Loss = 2.3754e-05, PNorm = 86.2447, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.249420
Epoch 250
Validation prc-auc = 0.249700
Epoch 251
Validation prc-auc = 0.250617
Epoch 252
Validation prc-auc = 0.250836
Epoch 253
Validation prc-auc = 0.254120
Epoch 254
Loss = 1.9915e-05, PNorm = 86.2511, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.253981
Epoch 255
Validation prc-auc = 0.252300
Epoch 256
Validation prc-auc = 0.252157
Epoch 257
Validation prc-auc = 0.251569
Epoch 258
Validation prc-auc = 0.251691
Epoch 259
Loss = 1.3458e-04, PNorm = 86.2563, GNorm = 0.0071, lr_0 = 6.4931e-04
Validation prc-auc = 0.254110
Epoch 260
Validation prc-auc = 0.254779
Epoch 261
Validation prc-auc = 0.258094
Epoch 262
Validation prc-auc = 0.258455
Epoch 263
Validation prc-auc = 0.258931
Epoch 264
Loss = 2.1682e-05, PNorm = 86.2614, GNorm = 0.0007, lr_0 = 6.4376e-04
Validation prc-auc = 0.258857
Epoch 265
Validation prc-auc = 0.258299
Epoch 266
Validation prc-auc = 0.257925
Epoch 267
Validation prc-auc = 0.251979
Epoch 268
Validation prc-auc = 0.252051
Epoch 269
Loss = 1.4931e-05, PNorm = 86.2663, GNorm = 0.0010, lr_0 = 6.3826e-04
Validation prc-auc = 0.251905
Epoch 270
Validation prc-auc = 0.251832
Epoch 271
Validation prc-auc = 0.251908
Epoch 272
Validation prc-auc = 0.254670
Epoch 273
Validation prc-auc = 0.254670
Epoch 274
Loss = 1.6231e-05, PNorm = 86.2704, GNorm = 0.0012, lr_0 = 6.3280e-04
Validation prc-auc = 0.254670
Epoch 275
Validation prc-auc = 0.254597
Epoch 276
Validation prc-auc = 0.254526
Epoch 277
Validation prc-auc = 0.258474
Epoch 278
Validation prc-auc = 0.262857
Epoch 279
Loss = 1.1756e-05, PNorm = 86.2735, GNorm = 0.0004, lr_0 = 6.2739e-04
Validation prc-auc = 0.262866
Epoch 280
Validation prc-auc = 0.259768
Epoch 281
Validation prc-auc = 0.260306
Epoch 282
Validation prc-auc = 0.266757
Epoch 283
Validation prc-auc = 0.264661
Epoch 284
Loss = 1.2980e-05, PNorm = 86.2767, GNorm = 0.0006, lr_0 = 6.2203e-04
Validation prc-auc = 0.264400
Epoch 285
Validation prc-auc = 0.265007
Epoch 286
Validation prc-auc = 0.268412
Epoch 287
Validation prc-auc = 0.267190
Epoch 288
Validation prc-auc = 0.266879
Epoch 289
Loss = 7.5815e-06, PNorm = 86.2803, GNorm = 0.0003, lr_0 = 6.1671e-04
Validation prc-auc = 0.263334
Epoch 290
Validation prc-auc = 0.260106
Epoch 291
Validation prc-auc = 0.263176
Epoch 292
Validation prc-auc = 0.266582
Epoch 293
Validation prc-auc = 0.266857
Epoch 294
Loss = 2.3784e-05, PNorm = 86.2831, GNorm = 0.0008, lr_0 = 6.1144e-04
Validation prc-auc = 0.270101
Epoch 295
Validation prc-auc = 0.274080
Epoch 296
Validation prc-auc = 0.274169
Epoch 297
Validation prc-auc = 0.273711
Epoch 298
Validation prc-auc = 0.273629
Epoch 299
Loss = 1.2231e-05, PNorm = 86.2869, GNorm = 0.0006, lr_0 = 6.0621e-04
Validation prc-auc = 0.273753
Model 0 best validation prc-auc = 0.465896 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.381926
Ensemble test prc-auc = 0.381926
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.435708
Epoch 1
Validation prc-auc = 0.441723
Epoch 2
Validation prc-auc = 0.454004
Epoch 3
Validation prc-auc = 0.425858
Epoch 4
Loss = 8.8239e-01, PNorm = 78.5518, GNorm = 11.6588, lr_0 = 6.5000e-04
Validation prc-auc = 0.422802
Epoch 5
Validation prc-auc = 0.446360
Epoch 6
Validation prc-auc = 0.497502
Epoch 7
Validation prc-auc = 0.450476
Epoch 8
Validation prc-auc = 0.460662
Epoch 9
Loss = 5.7273e-01, PNorm = 78.8056, GNorm = 6.4887, lr_0 = 9.9743e-04
Validation prc-auc = 0.499893
Epoch 10
Validation prc-auc = 0.474553
Epoch 11
Validation prc-auc = 0.500709
Epoch 12
Validation prc-auc = 0.465800
Epoch 13
Validation prc-auc = 0.422143
Epoch 14
Loss = 4.7215e-01, PNorm = 79.0723, GNorm = 0.8683, lr_0 = 9.8890e-04
Validation prc-auc = 0.437083
Epoch 15
Validation prc-auc = 0.427480
Epoch 16
Validation prc-auc = 0.303268
Epoch 17
Validation prc-auc = 0.290890
Epoch 18
Validation prc-auc = 0.290655
Epoch 19
Loss = 4.8168e-01, PNorm = 79.3140, GNorm = 3.5564, lr_0 = 9.8045e-04
Validation prc-auc = 0.310672
Epoch 20
Validation prc-auc = 0.382304
Epoch 21
Validation prc-auc = 0.399580
Epoch 22
Validation prc-auc = 0.486670
Epoch 23
Validation prc-auc = 0.478546
Epoch 24
Loss = 3.9170e-01, PNorm = 79.5683, GNorm = 3.1018, lr_0 = 9.7207e-04
Validation prc-auc = 0.485464
Epoch 25
Validation prc-auc = 0.476910
Epoch 26
Validation prc-auc = 0.524092
Epoch 27
Validation prc-auc = 0.502251
Epoch 28
Validation prc-auc = 0.455351
Epoch 29
Loss = 4.1243e-01, PNorm = 79.8351, GNorm = 4.8494, lr_0 = 9.6376e-04
Validation prc-auc = 0.429718
Epoch 30
Validation prc-auc = 0.390266
Epoch 31
Validation prc-auc = 0.278632
Epoch 32
Validation prc-auc = 0.292380
Epoch 33
Validation prc-auc = 0.277029
Epoch 34
Loss = 3.6675e-01, PNorm = 80.1210, GNorm = 1.2056, lr_0 = 9.5552e-04
Validation prc-auc = 0.275304
Epoch 35
Validation prc-auc = 0.303900
Epoch 36
Validation prc-auc = 0.271541
Epoch 37
Validation prc-auc = 0.298444
Epoch 38
Validation prc-auc = 0.299185
Epoch 39
Loss = 2.2963e-01, PNorm = 80.4620, GNorm = 2.7669, lr_0 = 9.4735e-04
Validation prc-auc = 0.307856
Epoch 40
Validation prc-auc = 0.325169
Epoch 41
Validation prc-auc = 0.332641
Epoch 42
Validation prc-auc = 0.330515
Epoch 43
Validation prc-auc = 0.345459
Epoch 44
Loss = 2.7264e-01, PNorm = 80.7752, GNorm = 3.9265, lr_0 = 9.3925e-04
Validation prc-auc = 0.325186
Epoch 45
Validation prc-auc = 0.303068
Epoch 46
Validation prc-auc = 0.229226
Epoch 47
Validation prc-auc = 0.296760
Epoch 48
Validation prc-auc = 0.261483
Epoch 49
Loss = 1.7006e-01, PNorm = 81.1059, GNorm = 1.2697, lr_0 = 9.3122e-04
Validation prc-auc = 0.292475
Epoch 50
Validation prc-auc = 0.330260
Epoch 51
Validation prc-auc = 0.218820
Epoch 52
Validation prc-auc = 0.273329
Epoch 53
Validation prc-auc = 0.242914
Epoch 54
Loss = 1.9472e-01, PNorm = 81.4353, GNorm = 2.1499, lr_0 = 9.2326e-04
Validation prc-auc = 0.160746
Epoch 55
Validation prc-auc = 0.272838
Epoch 56
Validation prc-auc = 0.338849
Epoch 57
Validation prc-auc = 0.333858
Epoch 58
Validation prc-auc = 0.228223
Epoch 59
Loss = 1.6898e-01, PNorm = 81.7784, GNorm = 1.6197, lr_0 = 9.1537e-04
Validation prc-auc = 0.205948
Epoch 60
Validation prc-auc = 0.240472
Epoch 61
Validation prc-auc = 0.278798
Epoch 62
Validation prc-auc = 0.212128
Epoch 63
Validation prc-auc = 0.317435
Epoch 64
Loss = 8.3831e-02, PNorm = 82.1227, GNorm = 1.7327, lr_0 = 9.0754e-04
Validation prc-auc = 0.353449
Epoch 65
Validation prc-auc = 0.340085
Epoch 66
Validation prc-auc = 0.318553
Epoch 67
Validation prc-auc = 0.204701
Epoch 68
Validation prc-auc = 0.195564
Epoch 69
Loss = 9.3059e-02, PNorm = 82.4415, GNorm = 0.6657, lr_0 = 8.9978e-04
Validation prc-auc = 0.233861
Epoch 70
Validation prc-auc = 0.260961
Epoch 71
Validation prc-auc = 0.266063
Epoch 72
Validation prc-auc = 0.252819
Epoch 73
Validation prc-auc = 0.293240
Epoch 74
Loss = 3.4455e-02, PNorm = 82.7423, GNorm = 0.8372, lr_0 = 8.9209e-04
Validation prc-auc = 0.244053
Epoch 75
Validation prc-auc = 0.246577
Epoch 76
Validation prc-auc = 0.280393
Epoch 77
Validation prc-auc = 0.281936
Epoch 78
Validation prc-auc = 0.299902
Epoch 79
Loss = 6.6255e-02, PNorm = 83.0010, GNorm = 0.5624, lr_0 = 8.8447e-04
Validation prc-auc = 0.290115
Epoch 80
Validation prc-auc = 0.287320
Epoch 81
Validation prc-auc = 0.240154
Epoch 82
Validation prc-auc = 0.213205
Epoch 83
Validation prc-auc = 0.240780
Epoch 84
Loss = 5.5168e-02, PNorm = 83.2307, GNorm = 1.3664, lr_0 = 8.7691e-04
Validation prc-auc = 0.233837
Epoch 85
Validation prc-auc = 0.207928
Epoch 86
Validation prc-auc = 0.197484
Epoch 87
Validation prc-auc = 0.212902
Epoch 88
Validation prc-auc = 0.221800
Epoch 89
Loss = 2.5878e-02, PNorm = 83.4308, GNorm = 0.4328, lr_0 = 8.6941e-04
Validation prc-auc = 0.265031
Epoch 90
Validation prc-auc = 0.231669
Epoch 91
Validation prc-auc = 0.237319
Epoch 92
Validation prc-auc = 0.284283
Epoch 93
Validation prc-auc = 0.304206
Epoch 94
Loss = 1.6962e-02, PNorm = 83.6489, GNorm = 1.1727, lr_0 = 8.6198e-04
Validation prc-auc = 0.343293
Epoch 95
Validation prc-auc = 0.345032
Epoch 96
Validation prc-auc = 0.342256
Epoch 97
Validation prc-auc = 0.291128
Epoch 98
Validation prc-auc = 0.210684
Epoch 99
Loss = 1.9196e-02, PNorm = 83.8497, GNorm = 0.4856, lr_0 = 8.5461e-04
Validation prc-auc = 0.192574
Epoch 100
Validation prc-auc = 0.209601
Epoch 101
Validation prc-auc = 0.250479
Epoch 102
Validation prc-auc = 0.278383
Epoch 103
Validation prc-auc = 0.275290
Epoch 104
Loss = 7.1857e-03, PNorm = 84.0357, GNorm = 0.2142, lr_0 = 8.4730e-04
Validation prc-auc = 0.264635
Epoch 105
Validation prc-auc = 0.254282
Epoch 106
Validation prc-auc = 0.264551
Epoch 107
Validation prc-auc = 0.294027
Epoch 108
Validation prc-auc = 0.286518
Epoch 109
Loss = 9.0730e-03, PNorm = 84.1814, GNorm = 0.8205, lr_0 = 8.4006e-04
Validation prc-auc = 0.270109
Epoch 110
Validation prc-auc = 0.268162
Epoch 111
Validation prc-auc = 0.259521
Epoch 112
Validation prc-auc = 0.205277
Epoch 113
Validation prc-auc = 0.166000
Epoch 114
Loss = 4.9406e-02, PNorm = 84.3463, GNorm = 1.9243, lr_0 = 8.3288e-04
Validation prc-auc = 0.212214
Epoch 115
Validation prc-auc = 0.263161
Epoch 116
Validation prc-auc = 0.279538
Epoch 117
Validation prc-auc = 0.219397
Epoch 118
Validation prc-auc = 0.253764
Epoch 119
Loss = 2.6383e-02, PNorm = 84.5484, GNorm = 0.2824, lr_0 = 8.2576e-04
Validation prc-auc = 0.273885
Epoch 120
Validation prc-auc = 0.246262
Epoch 121
Validation prc-auc = 0.191422
Epoch 122
Validation prc-auc = 0.216522
Epoch 123
Validation prc-auc = 0.219966
Epoch 124
Loss = 6.1804e-02, PNorm = 84.7972, GNorm = 1.0151, lr_0 = 8.1870e-04
Validation prc-auc = 0.185787
Epoch 125
Validation prc-auc = 0.200373
Epoch 126
Validation prc-auc = 0.242191
Epoch 127
Validation prc-auc = 0.240296
Epoch 128
Validation prc-auc = 0.178176
Epoch 129
Loss = 3.3523e-02, PNorm = 85.1395, GNorm = 0.5256, lr_0 = 8.1170e-04
Validation prc-auc = 0.204151
Epoch 130
Validation prc-auc = 0.218663
Epoch 131
Validation prc-auc = 0.236722
Epoch 132
Validation prc-auc = 0.215811
Epoch 133
Validation prc-auc = 0.202910
Epoch 134
Loss = 2.1253e-02, PNorm = 85.4255, GNorm = 0.4560, lr_0 = 8.0476e-04
Validation prc-auc = 0.196777
Epoch 135
Validation prc-auc = 0.205130
Epoch 136
Validation prc-auc = 0.227541
Epoch 137
Validation prc-auc = 0.252211
Epoch 138
Validation prc-auc = 0.245700
Epoch 139
Loss = 1.3113e-02, PNorm = 85.6583, GNorm = 0.0394, lr_0 = 7.9788e-04
Validation prc-auc = 0.222978
Epoch 140
Validation prc-auc = 0.207730
Epoch 141
Validation prc-auc = 0.198344
Epoch 142
Validation prc-auc = 0.196818
Epoch 143
Validation prc-auc = 0.201621
Epoch 144
Loss = 9.6422e-03, PNorm = 85.8139, GNorm = 0.3940, lr_0 = 7.9106e-04
Validation prc-auc = 0.215057
Epoch 145
Validation prc-auc = 0.227365
Epoch 146
Validation prc-auc = 0.252091
Epoch 147
Validation prc-auc = 0.275157
Epoch 148
Validation prc-auc = 0.289513
Epoch 149
Loss = 7.7328e-03, PNorm = 85.9599, GNorm = 0.0524, lr_0 = 7.8430e-04
Validation prc-auc = 0.274051
Epoch 150
Validation prc-auc = 0.253683
Epoch 151
Validation prc-auc = 0.198872
Epoch 152
Validation prc-auc = 0.189283
Epoch 153
Validation prc-auc = 0.191910
Epoch 154
Loss = 7.6692e-02, PNorm = 86.0930, GNorm = 1.6271, lr_0 = 7.7759e-04
Validation prc-auc = 0.228123
Epoch 155
Validation prc-auc = 0.272988
Epoch 156
Validation prc-auc = 0.286564
Epoch 157
Validation prc-auc = 0.280947
Epoch 158
Validation prc-auc = 0.238652
Epoch 159
Loss = 1.5534e-03, PNorm = 86.2253, GNorm = 0.0538, lr_0 = 7.7095e-04
Validation prc-auc = 0.203787
Epoch 160
Validation prc-auc = 0.174199
Epoch 161
Validation prc-auc = 0.165425
Epoch 162
Validation prc-auc = 0.167142
Epoch 163
Validation prc-auc = 0.173436
Epoch 164
Loss = 5.2604e-03, PNorm = 86.3228, GNorm = 0.0879, lr_0 = 7.6436e-04
Validation prc-auc = 0.182415
Epoch 165
Validation prc-auc = 0.194889
Epoch 166
Validation prc-auc = 0.218257
Epoch 167
Validation prc-auc = 0.238327
Epoch 168
Validation prc-auc = 0.247811
Epoch 169
Loss = 6.8510e-03, PNorm = 86.4010, GNorm = 0.3714, lr_0 = 7.5782e-04
Validation prc-auc = 0.245389
Epoch 170
Validation prc-auc = 0.243590
Epoch 171
Validation prc-auc = 0.238518
Epoch 172
Validation prc-auc = 0.241509
Epoch 173
Validation prc-auc = 0.232566
Epoch 174
Loss = 6.0077e-04, PNorm = 86.4580, GNorm = 0.0183, lr_0 = 7.5134e-04
Validation prc-auc = 0.228940
Epoch 175
Validation prc-auc = 0.230387
Epoch 176
Validation prc-auc = 0.217262
Epoch 177
Validation prc-auc = 0.226027
Epoch 178
Validation prc-auc = 0.234933
Epoch 179
Loss = 1.2234e-02, PNorm = 86.5019, GNorm = 0.3012, lr_0 = 7.4492e-04
Validation prc-auc = 0.237254
Epoch 180
Validation prc-auc = 0.231102
Epoch 181
Validation prc-auc = 0.231997
Epoch 182
Validation prc-auc = 0.225636
Epoch 183
Validation prc-auc = 0.225364
Epoch 184
Loss = 1.1444e-03, PNorm = 86.5539, GNorm = 0.0869, lr_0 = 7.3855e-04
Validation prc-auc = 0.225363
Epoch 185
Validation prc-auc = 0.220254
Epoch 186
Validation prc-auc = 0.226408
Epoch 187
Validation prc-auc = 0.230151
Epoch 188
Validation prc-auc = 0.243914
Epoch 189
Loss = 1.5982e-03, PNorm = 86.5929, GNorm = 0.0528, lr_0 = 7.3224e-04
Validation prc-auc = 0.244414
Epoch 190
Validation prc-auc = 0.243082
Epoch 191
Validation prc-auc = 0.243138
Epoch 192
Validation prc-auc = 0.255176
Epoch 193
Validation prc-auc = 0.255786
Epoch 194
Loss = 7.6292e-04, PNorm = 86.6298, GNorm = 0.0353, lr_0 = 7.2598e-04
Validation prc-auc = 0.257413
Epoch 195
Validation prc-auc = 0.245336
Epoch 196
Validation prc-auc = 0.245865
Epoch 197
Validation prc-auc = 0.245849
Epoch 198
Validation prc-auc = 0.251351
Epoch 199
Loss = 4.9669e-04, PNorm = 86.6535, GNorm = 0.0209, lr_0 = 7.1977e-04
Validation prc-auc = 0.251486
Epoch 200
Validation prc-auc = 0.252000
Epoch 201
Validation prc-auc = 0.246061
Epoch 202
Validation prc-auc = 0.246128
Epoch 203
Validation prc-auc = 0.246128
Epoch 204
Loss = 2.3363e-04, PNorm = 86.6683, GNorm = 0.0039, lr_0 = 7.1362e-04
Validation prc-auc = 0.246311
Epoch 205
Validation prc-auc = 0.246404
Epoch 206
Validation prc-auc = 0.246159
Epoch 207
Validation prc-auc = 0.246159
Epoch 208
Validation prc-auc = 0.246227
Epoch 209
Loss = 3.2975e-04, PNorm = 86.6775, GNorm = 0.0150, lr_0 = 7.0752e-04
Validation prc-auc = 0.245827
Epoch 210
Validation prc-auc = 0.243924
Epoch 211
Validation prc-auc = 0.243597
Epoch 212
Validation prc-auc = 0.243450
Epoch 213
Validation prc-auc = 0.243279
Epoch 214
Loss = 3.8837e-04, PNorm = 86.6858, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.238650
Epoch 215
Validation prc-auc = 0.238392
Epoch 216
Validation prc-auc = 0.238233
Epoch 217
Validation prc-auc = 0.237587
Epoch 218
Validation prc-auc = 0.237032
Epoch 219
Loss = 4.7102e-04, PNorm = 86.6975, GNorm = 0.0065, lr_0 = 6.9548e-04
Validation prc-auc = 0.236958
Epoch 220
Validation prc-auc = 0.236481
Epoch 221
Validation prc-auc = 0.236329
Epoch 222
Validation prc-auc = 0.236706
Epoch 223
Validation prc-auc = 0.236978
Epoch 224
Loss = 1.0850e-04, PNorm = 86.7101, GNorm = 0.0023, lr_0 = 6.8953e-04
Validation prc-auc = 0.241532
Epoch 225
Validation prc-auc = 0.241425
Epoch 226
Validation prc-auc = 0.241691
Epoch 227
Validation prc-auc = 0.241903
Epoch 228
Validation prc-auc = 0.241592
Epoch 229
Loss = 1.9321e-04, PNorm = 86.7197, GNorm = 0.0052, lr_0 = 6.8364e-04
Validation prc-auc = 0.241513
Epoch 230
Validation prc-auc = 0.241515
Epoch 231
Validation prc-auc = 0.241367
Epoch 232
Validation prc-auc = 0.240913
Epoch 233
Validation prc-auc = 0.241020
Epoch 234
Loss = 1.1889e-04, PNorm = 86.7272, GNorm = 0.0030, lr_0 = 6.7779e-04
Validation prc-auc = 0.241020
Epoch 235
Validation prc-auc = 0.241082
Epoch 236
Validation prc-auc = 0.240905
Epoch 237
Validation prc-auc = 0.240910
Epoch 238
Validation prc-auc = 0.240509
Epoch 239
Loss = 1.9137e-04, PNorm = 86.7349, GNorm = 0.0056, lr_0 = 6.7200e-04
Validation prc-auc = 0.240506
Epoch 240
Validation prc-auc = 0.240262
Epoch 241
Validation prc-auc = 0.240089
Epoch 242
Validation prc-auc = 0.240016
Epoch 243
Validation prc-auc = 0.240016
Epoch 244
Loss = 1.2605e-04, PNorm = 86.7433, GNorm = 0.0025, lr_0 = 6.6625e-04
Validation prc-auc = 0.246301
Epoch 245
Validation prc-auc = 0.246373
Epoch 246
Validation prc-auc = 0.240089
Epoch 247
Validation prc-auc = 0.240089
Epoch 248
Validation prc-auc = 0.239977
Epoch 249
Loss = 1.0577e-04, PNorm = 86.7513, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.240178
Epoch 250
Validation prc-auc = 0.240178
Epoch 251
Validation prc-auc = 0.240176
Epoch 252
Validation prc-auc = 0.239800
Epoch 253
Validation prc-auc = 0.240341
Epoch 254
Loss = 4.3154e-05, PNorm = 86.7612, GNorm = 0.0018, lr_0 = 6.5491e-04
Validation prc-auc = 0.240416
Epoch 255
Validation prc-auc = 0.240238
Epoch 256
Validation prc-auc = 0.240495
Epoch 257
Validation prc-auc = 0.240490
Epoch 258
Validation prc-auc = 0.240557
Epoch 259
Loss = 5.0080e-05, PNorm = 86.7691, GNorm = 0.0008, lr_0 = 6.4931e-04
Validation prc-auc = 0.240635
Epoch 260
Validation prc-auc = 0.240310
Epoch 261
Validation prc-auc = 0.240373
Epoch 262
Validation prc-auc = 0.240293
Epoch 263
Validation prc-auc = 0.240440
Epoch 264
Loss = 5.8278e-05, PNorm = 86.7764, GNorm = 0.0018, lr_0 = 6.4376e-04
Validation prc-auc = 0.240860
Epoch 265
Validation prc-auc = 0.241244
Epoch 266
Validation prc-auc = 0.241244
Epoch 267
Validation prc-auc = 0.241372
Epoch 268
Validation prc-auc = 0.241298
Epoch 269
Loss = 1.0803e-04, PNorm = 86.7852, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.241465
Epoch 270
Validation prc-auc = 0.241605
Epoch 271
Validation prc-auc = 0.241605
Epoch 272
Validation prc-auc = 0.241691
Epoch 273
Validation prc-auc = 0.241691
Epoch 274
Loss = 1.1218e-04, PNorm = 86.7927, GNorm = 0.0035, lr_0 = 6.3280e-04
Validation prc-auc = 0.241691
Epoch 275
Validation prc-auc = 0.243121
Epoch 276
Validation prc-auc = 0.243121
Epoch 277
Validation prc-auc = 0.242998
Epoch 278
Validation prc-auc = 0.242833
Epoch 279
Loss = 1.2764e-04, PNorm = 86.8023, GNorm = 0.0025, lr_0 = 6.2739e-04
Validation prc-auc = 0.243086
Epoch 280
Validation prc-auc = 0.243345
Epoch 281
Validation prc-auc = 0.243438
Epoch 282
Validation prc-auc = 0.243802
Epoch 283
Validation prc-auc = 0.243802
Epoch 284
Loss = 1.4733e-04, PNorm = 86.8134, GNorm = 0.0133, lr_0 = 6.2203e-04
Validation prc-auc = 0.243856
Epoch 285
Validation prc-auc = 0.243856
Epoch 286
Validation prc-auc = 0.244168
Epoch 287
Validation prc-auc = 0.244098
Epoch 288
Validation prc-auc = 0.244098
Epoch 289
Loss = 3.4529e-05, PNorm = 86.8239, GNorm = 0.0006, lr_0 = 6.1671e-04
Validation prc-auc = 0.243966
Epoch 290
Validation prc-auc = 0.243693
Epoch 291
Validation prc-auc = 0.243623
Epoch 292
Validation prc-auc = 0.243940
Epoch 293
Validation prc-auc = 0.244010
Epoch 294
Loss = 3.3168e-05, PNorm = 86.8315, GNorm = 0.0009, lr_0 = 6.1144e-04
Validation prc-auc = 0.250264
Epoch 295
Validation prc-auc = 0.249965
Epoch 296
Validation prc-auc = 0.249854
Epoch 297
Validation prc-auc = 0.249821
Epoch 298
Validation prc-auc = 0.249514
Epoch 299
Loss = 3.6395e-05, PNorm = 86.8380, GNorm = 0.0027, lr_0 = 6.0621e-04
Validation prc-auc = 0.249688
Model 0 best validation prc-auc = 0.524092 on epoch 26
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.512151
Ensemble test prc-auc = 0.512151
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.451981
Epoch 1
Validation prc-auc = 0.446994
Epoch 2
Validation prc-auc = 0.456972
Epoch 3
Validation prc-auc = 0.434877
Epoch 4
Loss = 7.3300e-01, PNorm = 78.5513, GNorm = 15.6126, lr_0 = 6.5000e-04
Validation prc-auc = 0.447210
Epoch 5
Validation prc-auc = 0.363943
Epoch 6
Validation prc-auc = 0.448088
Epoch 7
Validation prc-auc = 0.492841
Epoch 8
Validation prc-auc = 0.466519
Epoch 9
Loss = 6.7786e-01, PNorm = 78.8123, GNorm = 6.3473, lr_0 = 9.9743e-04
Validation prc-auc = 0.485870
Epoch 10
Validation prc-auc = 0.319707
Epoch 11
Validation prc-auc = 0.311744
Epoch 12
Validation prc-auc = 0.295678
Epoch 13
Validation prc-auc = 0.431364
Epoch 14
Loss = 5.5333e-01, PNorm = 79.1009, GNorm = 3.2639, lr_0 = 9.8890e-04
Validation prc-auc = 0.291894
Epoch 15
Validation prc-auc = 0.298993
Epoch 16
Validation prc-auc = 0.320662
Epoch 17
Validation prc-auc = 0.412705
Epoch 18
Validation prc-auc = 0.392252
Epoch 19
Loss = 4.9499e-01, PNorm = 79.3513, GNorm = 1.1391, lr_0 = 9.8045e-04
Validation prc-auc = 0.398266
Epoch 20
Validation prc-auc = 0.415484
Epoch 21
Validation prc-auc = 0.302287
Epoch 22
Validation prc-auc = 0.241544
Epoch 23
Validation prc-auc = 0.332738
Epoch 24
Loss = 4.7328e-01, PNorm = 79.6106, GNorm = 4.8199, lr_0 = 9.7207e-04
Validation prc-auc = 0.361985
Epoch 25
Validation prc-auc = 0.338897
Epoch 26
Validation prc-auc = 0.362187
Epoch 27
Validation prc-auc = 0.343060
Epoch 28
Validation prc-auc = 0.315752
Epoch 29
Loss = 4.4372e-01, PNorm = 79.9035, GNorm = 5.2715, lr_0 = 9.6376e-04
Validation prc-auc = 0.294318
Epoch 30
Validation prc-auc = 0.390667
Epoch 31
Validation prc-auc = 0.411278
Epoch 32
Validation prc-auc = 0.419696
Epoch 33
Validation prc-auc = 0.427415
Epoch 34
Loss = 4.3009e-01, PNorm = 80.2022, GNorm = 3.1468, lr_0 = 9.5552e-04
Validation prc-auc = 0.421484
Epoch 35
Validation prc-auc = 0.332690
Epoch 36
Validation prc-auc = 0.343598
Epoch 37
Validation prc-auc = 0.348775
Epoch 38
Validation prc-auc = 0.382581
Epoch 39
Loss = 3.1726e-01, PNorm = 80.4962, GNorm = 0.8414, lr_0 = 9.4735e-04
Validation prc-auc = 0.381942
Epoch 40
Validation prc-auc = 0.362318
Epoch 41
Validation prc-auc = 0.355381
Epoch 42
Validation prc-auc = 0.322578
Epoch 43
Validation prc-auc = 0.332220
Epoch 44
Loss = 2.5246e-01, PNorm = 80.8630, GNorm = 1.3613, lr_0 = 9.3925e-04
Validation prc-auc = 0.316774
Epoch 45
Validation prc-auc = 0.297024
Epoch 46
Validation prc-auc = 0.283733
Epoch 47
Validation prc-auc = 0.291580
Epoch 48
Validation prc-auc = 0.286660
Epoch 49
Loss = 1.8035e-01, PNorm = 81.2976, GNorm = 1.0920, lr_0 = 9.3122e-04
Validation prc-auc = 0.272602
Epoch 50
Validation prc-auc = 0.287387
Epoch 51
Validation prc-auc = 0.266252
Epoch 52
Validation prc-auc = 0.163727
Epoch 53
Validation prc-auc = 0.213129
Epoch 54
Loss = 1.4254e-01, PNorm = 81.7439, GNorm = 0.5898, lr_0 = 9.2326e-04
Validation prc-auc = 0.231894
Epoch 55
Validation prc-auc = 0.380404
Epoch 56
Validation prc-auc = 0.350888
Epoch 57
Validation prc-auc = 0.358191
Epoch 58
Validation prc-auc = 0.363654
Epoch 59
Loss = 1.2802e-01, PNorm = 82.1729, GNorm = 1.3726, lr_0 = 9.1537e-04
Validation prc-auc = 0.296677
Epoch 60
Validation prc-auc = 0.280951
Epoch 61
Validation prc-auc = 0.272732
Epoch 62
Validation prc-auc = 0.256215
Epoch 63
Validation prc-auc = 0.261775
Epoch 64
Loss = 6.7330e-02, PNorm = 82.5714, GNorm = 1.0194, lr_0 = 9.0754e-04
Validation prc-auc = 0.266497
Epoch 65
Validation prc-auc = 0.302952
Epoch 66
Validation prc-auc = 0.313754
Epoch 67
Validation prc-auc = 0.313195
Epoch 68
Validation prc-auc = 0.331574
Epoch 69
Loss = 9.0878e-02, PNorm = 82.9257, GNorm = 2.1943, lr_0 = 8.9978e-04
Validation prc-auc = 0.300863
Epoch 70
Validation prc-auc = 0.286141
Epoch 71
Validation prc-auc = 0.296993
Epoch 72
Validation prc-auc = 0.281545
Epoch 73
Validation prc-auc = 0.276575
Epoch 74
Loss = 2.3740e-02, PNorm = 83.2514, GNorm = 0.3164, lr_0 = 8.9209e-04
Validation prc-auc = 0.282224
Epoch 75
Validation prc-auc = 0.279330
Epoch 76
Validation prc-auc = 0.263277
Epoch 77
Validation prc-auc = 0.255019
Epoch 78
Validation prc-auc = 0.271515
Epoch 79
Loss = 4.1753e-02, PNorm = 83.5512, GNorm = 0.4995, lr_0 = 8.8447e-04
Validation prc-auc = 0.315346
Epoch 80
Validation prc-auc = 0.330518
Epoch 81
Validation prc-auc = 0.306628
Epoch 82
Validation prc-auc = 0.302877
Epoch 83
Validation prc-auc = 0.333172
Epoch 84
Loss = 1.0037e-02, PNorm = 83.8114, GNorm = 0.1544, lr_0 = 8.7691e-04
Validation prc-auc = 0.332299
Epoch 85
Validation prc-auc = 0.334477
Epoch 86
Validation prc-auc = 0.329965
Epoch 87
Validation prc-auc = 0.270618
Epoch 88
Validation prc-auc = 0.278125
Epoch 89
Loss = 1.0980e-01, PNorm = 84.0138, GNorm = 1.2087, lr_0 = 8.6941e-04
Validation prc-auc = 0.307675
Epoch 90
Validation prc-auc = 0.331740
Epoch 91
Validation prc-auc = 0.351743
Epoch 92
Validation prc-auc = 0.350187
Epoch 93
Validation prc-auc = 0.318731
Epoch 94
Loss = 1.5660e-01, PNorm = 84.2226, GNorm = 1.2927, lr_0 = 8.6198e-04
Validation prc-auc = 0.326551
Epoch 95
Validation prc-auc = 0.291725
Epoch 96
Validation prc-auc = 0.195668
Epoch 97
Validation prc-auc = 0.184707
Epoch 98
Validation prc-auc = 0.232739
Epoch 99
Loss = 6.2176e-02, PNorm = 84.4634, GNorm = 2.4517, lr_0 = 8.5461e-04
Validation prc-auc = 0.248480
Epoch 100
Validation prc-auc = 0.179112
Epoch 101
Validation prc-auc = 0.201404
Epoch 102
Validation prc-auc = 0.247289
Epoch 103
Validation prc-auc = 0.251840
Epoch 104
Loss = 3.0495e-02, PNorm = 84.7787, GNorm = 2.0164, lr_0 = 8.4730e-04
Validation prc-auc = 0.284531
Epoch 105
Validation prc-auc = 0.336537
Epoch 106
Validation prc-auc = 0.356780
Epoch 107
Validation prc-auc = 0.359496
Epoch 108
Validation prc-auc = 0.332561
Epoch 109
Loss = 2.3595e-02, PNorm = 85.1265, GNorm = 0.2145, lr_0 = 8.4006e-04
Validation prc-auc = 0.289069
Epoch 110
Validation prc-auc = 0.262988
Epoch 111
Validation prc-auc = 0.271154
Epoch 112
Validation prc-auc = 0.264520
Epoch 113
Validation prc-auc = 0.269497
Epoch 114
Loss = 2.2038e-01, PNorm = 85.4287, GNorm = 2.4526, lr_0 = 8.3288e-04
Validation prc-auc = 0.244269
Epoch 115
Validation prc-auc = 0.184818
Epoch 116
Validation prc-auc = 0.165806
Epoch 117
Validation prc-auc = 0.254500
Epoch 118
Validation prc-auc = 0.312299
Epoch 119
Loss = 5.2687e-02, PNorm = 85.7742, GNorm = 0.7315, lr_0 = 8.2576e-04
Validation prc-auc = 0.274522
Epoch 120
Validation prc-auc = 0.226121
Epoch 121
Validation prc-auc = 0.215974
Epoch 122
Validation prc-auc = 0.204843
Epoch 123
Validation prc-auc = 0.243778
Epoch 124
Loss = 6.1667e-02, PNorm = 86.0668, GNorm = 2.7278, lr_0 = 8.1870e-04
Validation prc-auc = 0.215740
Epoch 125
Validation prc-auc = 0.204278
Epoch 126
Validation prc-auc = 0.237765
Epoch 127
Validation prc-auc = 0.236781
Epoch 128
Validation prc-auc = 0.202069
Epoch 129
Loss = 3.9290e-02, PNorm = 86.4271, GNorm = 1.4355, lr_0 = 8.1170e-04
Validation prc-auc = 0.171611
Epoch 130
Validation prc-auc = 0.188763
Epoch 131
Validation prc-auc = 0.273622
Epoch 132
Validation prc-auc = 0.285016
Epoch 133
Validation prc-auc = 0.295705
Epoch 134
Loss = 2.2487e-02, PNorm = 86.7805, GNorm = 0.6677, lr_0 = 8.0476e-04
Validation prc-auc = 0.295033
Epoch 135
Validation prc-auc = 0.313064
Epoch 136
Validation prc-auc = 0.314268
Epoch 137
Validation prc-auc = 0.305511
Epoch 138
Validation prc-auc = 0.300970
Epoch 139
Loss = 1.4917e-02, PNorm = 87.0262, GNorm = 0.8027, lr_0 = 7.9788e-04
Validation prc-auc = 0.289984
Epoch 140
Validation prc-auc = 0.289688
Epoch 141
Validation prc-auc = 0.272827
Epoch 142
Validation prc-auc = 0.243509
Epoch 143
Validation prc-auc = 0.229837
Epoch 144
Loss = 1.1503e-02, PNorm = 87.2025, GNorm = 0.1404, lr_0 = 7.9106e-04
Validation prc-auc = 0.200323
Epoch 145
Validation prc-auc = 0.179604
Epoch 146
Validation prc-auc = 0.177672
Epoch 147
Validation prc-auc = 0.190052
Epoch 148
Validation prc-auc = 0.203944
Epoch 149
Loss = 1.1095e-02, PNorm = 87.3288, GNorm = 0.3025, lr_0 = 7.8430e-04
Validation prc-auc = 0.204917
Epoch 150
Validation prc-auc = 0.209018
Epoch 151
Validation prc-auc = 0.209511
Epoch 152
Validation prc-auc = 0.204461
Epoch 153
Validation prc-auc = 0.237384
Epoch 154
Loss = 3.4870e-03, PNorm = 87.4270, GNorm = 0.1493, lr_0 = 7.7759e-04
Validation prc-auc = 0.246913
Epoch 155
Validation prc-auc = 0.250661
Epoch 156
Validation prc-auc = 0.259806
Epoch 157
Validation prc-auc = 0.263446
Epoch 158
Validation prc-auc = 0.269533
Epoch 159
Loss = 2.8702e-03, PNorm = 87.5061, GNorm = 0.0384, lr_0 = 7.7095e-04
Validation prc-auc = 0.274294
Epoch 160
Validation prc-auc = 0.261164
Epoch 161
Validation prc-auc = 0.260509
Epoch 162
Validation prc-auc = 0.257529
Epoch 163
Validation prc-auc = 0.237820
Epoch 164
Loss = 2.2486e-03, PNorm = 87.5652, GNorm = 0.0561, lr_0 = 7.6436e-04
Validation prc-auc = 0.234714
Epoch 165
Validation prc-auc = 0.223052
Epoch 166
Validation prc-auc = 0.230105
Epoch 167
Validation prc-auc = 0.230004
Epoch 168
Validation prc-auc = 0.226148
Epoch 169
Loss = 1.4940e-03, PNorm = 87.6145, GNorm = 0.0345, lr_0 = 7.5782e-04
Validation prc-auc = 0.225737
Epoch 170
Validation prc-auc = 0.217345
Epoch 171
Validation prc-auc = 0.217235
Epoch 172
Validation prc-auc = 0.217558
Epoch 173
Validation prc-auc = 0.223173
Epoch 174
Loss = 8.7847e-04, PNorm = 87.6570, GNorm = 0.0279, lr_0 = 7.5134e-04
Validation prc-auc = 0.222471
Epoch 175
Validation prc-auc = 0.222174
Epoch 176
Validation prc-auc = 0.220916
Epoch 177
Validation prc-auc = 0.220301
Epoch 178
Validation prc-auc = 0.220193
Epoch 179
Loss = 4.9069e-04, PNorm = 87.6854, GNorm = 0.0094, lr_0 = 7.4492e-04
Validation prc-auc = 0.219958
Epoch 180
Validation prc-auc = 0.219895
Epoch 181
Validation prc-auc = 0.221900
Epoch 182
Validation prc-auc = 0.222981
Epoch 183
Validation prc-auc = 0.231372
Epoch 184
Loss = 1.1542e-03, PNorm = 87.7007, GNorm = 0.0182, lr_0 = 7.3855e-04
Validation prc-auc = 0.233313
Epoch 185
Validation prc-auc = 0.240918
Epoch 186
Validation prc-auc = 0.242466
Epoch 187
Validation prc-auc = 0.242526
Epoch 188
Validation prc-auc = 0.244635
Epoch 189
Loss = 2.0666e-03, PNorm = 87.7300, GNorm = 0.0243, lr_0 = 7.3224e-04
Validation prc-auc = 0.252570
Epoch 190
Validation prc-auc = 0.265322
Epoch 191
Validation prc-auc = 0.261996
Epoch 192
Validation prc-auc = 0.264496
Epoch 193
Validation prc-auc = 0.255808
Epoch 194
Loss = 4.4080e-03, PNorm = 87.7632, GNorm = 0.4909, lr_0 = 7.2598e-04
Validation prc-auc = 0.243308
Epoch 195
Validation prc-auc = 0.249994
Epoch 196
Validation prc-auc = 0.238423
Epoch 197
Validation prc-auc = 0.234646
Epoch 198
Validation prc-auc = 0.247178
Epoch 199
Loss = 7.2347e-04, PNorm = 87.7936, GNorm = 0.0180, lr_0 = 7.1977e-04
Validation prc-auc = 0.246259
Epoch 200
Validation prc-auc = 0.251045
Epoch 201
Validation prc-auc = 0.257668
Epoch 202
Validation prc-auc = 0.257949
Epoch 203
Validation prc-auc = 0.260627
Epoch 204
Loss = 7.0931e-04, PNorm = 87.8198, GNorm = 0.0246, lr_0 = 7.1362e-04
Validation prc-auc = 0.261541
Epoch 205
Validation prc-auc = 0.261455
Epoch 206
Validation prc-auc = 0.264477
Epoch 207
Validation prc-auc = 0.262961
Epoch 208
Validation prc-auc = 0.263147
Epoch 209
Loss = 4.5923e-04, PNorm = 87.8424, GNorm = 0.0146, lr_0 = 7.0752e-04
Validation prc-auc = 0.260400
Epoch 210
Validation prc-auc = 0.254651
Epoch 211
Validation prc-auc = 0.254685
Epoch 212
Validation prc-auc = 0.254610
Epoch 213
Validation prc-auc = 0.255038
Epoch 214
Loss = 3.3965e-04, PNorm = 87.8602, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.255038
Epoch 215
Validation prc-auc = 0.254281
Epoch 216
Validation prc-auc = 0.253146
Epoch 217
Validation prc-auc = 0.253506
Epoch 218
Validation prc-auc = 0.254100
Epoch 219
Loss = 6.8978e-04, PNorm = 87.8762, GNorm = 0.0085, lr_0 = 6.9548e-04
Validation prc-auc = 0.252629
Epoch 220
Validation prc-auc = 0.261809
Epoch 221
Validation prc-auc = 0.268167
Epoch 222
Validation prc-auc = 0.268249
Epoch 223
Validation prc-auc = 0.268976
Epoch 224
Loss = 4.5019e-04, PNorm = 87.8891, GNorm = 0.0198, lr_0 = 6.8953e-04
Validation prc-auc = 0.269516
Epoch 225
Validation prc-auc = 0.274278
Epoch 226
Validation prc-auc = 0.274109
Epoch 227
Validation prc-auc = 0.273516
Epoch 228
Validation prc-auc = 0.265833
Epoch 229
Loss = 1.7040e-04, PNorm = 87.9016, GNorm = 0.0061, lr_0 = 6.8364e-04
Validation prc-auc = 0.270684
Epoch 230
Validation prc-auc = 0.270185
Epoch 231
Validation prc-auc = 0.270694
Epoch 232
Validation prc-auc = 0.270751
Epoch 233
Validation prc-auc = 0.270631
Epoch 234
Loss = 1.4315e-04, PNorm = 87.9119, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.270455
Epoch 235
Validation prc-auc = 0.270370
Epoch 236
Validation prc-auc = 0.270231
Epoch 237
Validation prc-auc = 0.270231
Epoch 238
Validation prc-auc = 0.270231
Epoch 239
Loss = 1.9387e-04, PNorm = 87.9192, GNorm = 0.0094, lr_0 = 6.7200e-04
Validation prc-auc = 0.269676
Epoch 240
Validation prc-auc = 0.269793
Epoch 241
Validation prc-auc = 0.269793
Epoch 242
Validation prc-auc = 0.269793
Epoch 243
Validation prc-auc = 0.269793
Epoch 244
Loss = 1.6600e-04, PNorm = 87.9262, GNorm = 0.0035, lr_0 = 6.6625e-04
Validation prc-auc = 0.269894
Epoch 245
Validation prc-auc = 0.270105
Epoch 246
Validation prc-auc = 0.276827
Epoch 247
Validation prc-auc = 0.276363
Epoch 248
Validation prc-auc = 0.276339
Epoch 249
Loss = 1.9239e-04, PNorm = 87.9332, GNorm = 0.0101, lr_0 = 6.6056e-04
Validation prc-auc = 0.275668
Epoch 250
Validation prc-auc = 0.275034
Epoch 251
Validation prc-auc = 0.268477
Epoch 252
Validation prc-auc = 0.268418
Epoch 253
Validation prc-auc = 0.268747
Epoch 254
Loss = 1.3926e-04, PNorm = 87.9405, GNorm = 0.0027, lr_0 = 6.5491e-04
Validation prc-auc = 0.268457
Epoch 255
Validation prc-auc = 0.268238
Epoch 256
Validation prc-auc = 0.268125
Epoch 257
Validation prc-auc = 0.268185
Epoch 258
Validation prc-auc = 0.275545
Epoch 259
Loss = 9.2985e-05, PNorm = 87.9478, GNorm = 0.0034, lr_0 = 6.4931e-04
Validation prc-auc = 0.275510
Epoch 260
Validation prc-auc = 0.275650
Epoch 261
Validation prc-auc = 0.276198
Epoch 262
Validation prc-auc = 0.276275
Epoch 263
Validation prc-auc = 0.276275
Epoch 264
Loss = 1.4267e-04, PNorm = 87.9554, GNorm = 0.0052, lr_0 = 6.4376e-04
Validation prc-auc = 0.276192
Epoch 265
Validation prc-auc = 0.276192
Epoch 266
Validation prc-auc = 0.276054
Epoch 267
Validation prc-auc = 0.276054
Epoch 268
Validation prc-auc = 0.276232
Epoch 269
Loss = 1.1702e-04, PNorm = 87.9637, GNorm = 0.0053, lr_0 = 6.3826e-04
Validation prc-auc = 0.276206
Epoch 270
Validation prc-auc = 0.276298
Epoch 271
Validation prc-auc = 0.276481
Epoch 272
Validation prc-auc = 0.276481
Epoch 273
Validation prc-auc = 0.276481
Epoch 274
Loss = 3.3947e-05, PNorm = 87.9706, GNorm = 0.0006, lr_0 = 6.3280e-04
Validation prc-auc = 0.276481
Epoch 275
Validation prc-auc = 0.276337
Epoch 276
Validation prc-auc = 0.276512
Epoch 277
Validation prc-auc = 0.276575
Epoch 278
Validation prc-auc = 0.275938
Epoch 279
Loss = 6.3084e-05, PNorm = 87.9764, GNorm = 0.0055, lr_0 = 6.2739e-04
Validation prc-auc = 0.275856
Epoch 280
Validation prc-auc = 0.275670
Epoch 281
Validation prc-auc = 0.275638
Epoch 282
Validation prc-auc = 0.264682
Epoch 283
Validation prc-auc = 0.265450
Epoch 284
Loss = 7.0341e-05, PNorm = 87.9810, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.265515
Epoch 285
Validation prc-auc = 0.265515
Epoch 286
Validation prc-auc = 0.265436
Epoch 287
Validation prc-auc = 0.275830
Epoch 288
Validation prc-auc = 0.275823
Epoch 289
Loss = 4.6313e-05, PNorm = 87.9878, GNorm = 0.0019, lr_0 = 6.1671e-04
Validation prc-auc = 0.275288
Epoch 290
Validation prc-auc = 0.274733
Epoch 291
Validation prc-auc = 0.274733
Epoch 292
Validation prc-auc = 0.274351
Epoch 293
Validation prc-auc = 0.274579
Epoch 294
Loss = 1.4626e-04, PNorm = 87.9938, GNorm = 0.0015, lr_0 = 6.1144e-04
Validation prc-auc = 0.274670
Epoch 295
Validation prc-auc = 0.274853
Epoch 296
Validation prc-auc = 0.275408
Epoch 297
Validation prc-auc = 0.276237
Epoch 298
Validation prc-auc = 0.277788
Epoch 299
Loss = 4.2091e-05, PNorm = 87.9985, GNorm = 0.0032, lr_0 = 6.0621e-04
Validation prc-auc = 0.277967
Model 0 best validation prc-auc = 0.492841 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.437878
Ensemble test prc-auc = 0.437878
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.447435
Epoch 1
Validation prc-auc = 0.455436
Epoch 2
Validation prc-auc = 0.437530
Epoch 3
Validation prc-auc = 0.433102
Epoch 4
Loss = 6.7931e-01, PNorm = 78.5478, GNorm = 5.8820, lr_0 = 6.5000e-04
Validation prc-auc = 0.440586
Epoch 5
Validation prc-auc = 0.497692
Epoch 6
Validation prc-auc = 0.424765
Epoch 7
Validation prc-auc = 0.395213
Epoch 8
Validation prc-auc = 0.294183
Epoch 9
Loss = 5.2405e-01, PNorm = 78.7933, GNorm = 6.3021, lr_0 = 9.9743e-04
Validation prc-auc = 0.335632
Epoch 10
Validation prc-auc = 0.283611
Epoch 11
Validation prc-auc = 0.289981
Epoch 12
Validation prc-auc = 0.306816
Epoch 13
Validation prc-auc = 0.280692
Epoch 14
Loss = 5.3578e-01, PNorm = 79.0958, GNorm = 5.6050, lr_0 = 9.8890e-04
Validation prc-auc = 0.455066
Epoch 15
Validation prc-auc = 0.476875
Epoch 16
Validation prc-auc = 0.488546
Epoch 17
Validation prc-auc = 0.270603
Epoch 18
Validation prc-auc = 0.275675
Epoch 19
Loss = 5.5998e-01, PNorm = 79.4052, GNorm = 11.0964, lr_0 = 9.8045e-04
Validation prc-auc = 0.270895
Epoch 20
Validation prc-auc = 0.401254
Epoch 21
Validation prc-auc = 0.414864
Epoch 22
Validation prc-auc = 0.423959
Epoch 23
Validation prc-auc = 0.415277
Epoch 24
Loss = 4.8958e-01, PNorm = 79.7725, GNorm = 2.0160, lr_0 = 9.7207e-04
Validation prc-auc = 0.295220
Epoch 25
Validation prc-auc = 0.323758
Epoch 26
Validation prc-auc = 0.364850
Epoch 27
Validation prc-auc = 0.338250
Epoch 28
Validation prc-auc = 0.334354
Epoch 29
Loss = 3.8850e-01, PNorm = 80.1224, GNorm = 3.6337, lr_0 = 9.6376e-04
Validation prc-auc = 0.319221
Epoch 30
Validation prc-auc = 0.292483
Epoch 31
Validation prc-auc = 0.276862
Epoch 32
Validation prc-auc = 0.329416
Epoch 33
Validation prc-auc = 0.317479
Epoch 34
Loss = 2.8719e-01, PNorm = 80.4957, GNorm = 1.0380, lr_0 = 9.5552e-04
Validation prc-auc = 0.358420
Epoch 35
Validation prc-auc = 0.317927
Epoch 36
Validation prc-auc = 0.296096
Epoch 37
Validation prc-auc = 0.275368
Epoch 38
Validation prc-auc = 0.245756
Epoch 39
Loss = 3.0297e-01, PNorm = 80.8953, GNorm = 1.1152, lr_0 = 9.4735e-04
Validation prc-auc = 0.261501
Epoch 40
Validation prc-auc = 0.315310
Epoch 41
Validation prc-auc = 0.302531
Epoch 42
Validation prc-auc = 0.318977
Epoch 43
Validation prc-auc = 0.310490
Epoch 44
Loss = 1.7789e-01, PNorm = 81.3788, GNorm = 2.3393, lr_0 = 9.3925e-04
Validation prc-auc = 0.288724
Epoch 45
Validation prc-auc = 0.273710
Epoch 46
Validation prc-auc = 0.309345
Epoch 47
Validation prc-auc = 0.295845
Epoch 48
Validation prc-auc = 0.303109
Epoch 49
Loss = 3.0189e-01, PNorm = 81.9124, GNorm = 3.0181, lr_0 = 9.3122e-04
Validation prc-auc = 0.320837
Epoch 50
Validation prc-auc = 0.324325
Epoch 51
Validation prc-auc = 0.312761
Epoch 52
Validation prc-auc = 0.308602
Epoch 53
Validation prc-auc = 0.256017
Epoch 54
Loss = 9.6265e-02, PNorm = 82.4519, GNorm = 1.2409, lr_0 = 9.2326e-04
Validation prc-auc = 0.263593
Epoch 55
Validation prc-auc = 0.248617
Epoch 56
Validation prc-auc = 0.235027
Epoch 57
Validation prc-auc = 0.304428
Epoch 58
Validation prc-auc = 0.309252
Epoch 59
Loss = 5.3974e-02, PNorm = 82.9363, GNorm = 0.8254, lr_0 = 9.1537e-04
Validation prc-auc = 0.307894
Epoch 60
Validation prc-auc = 0.308851
Epoch 61
Validation prc-auc = 0.295125
Epoch 62
Validation prc-auc = 0.274480
Epoch 63
Validation prc-auc = 0.265507
Epoch 64
Loss = 1.0481e-01, PNorm = 83.3483, GNorm = 0.3695, lr_0 = 9.0754e-04
Validation prc-auc = 0.220221
Epoch 65
Validation prc-auc = 0.294121
Epoch 66
Validation prc-auc = 0.297513
Epoch 67
Validation prc-auc = 0.274874
Epoch 68
Validation prc-auc = 0.244755
Epoch 69
Loss = 6.7708e-02, PNorm = 83.7869, GNorm = 2.4059, lr_0 = 8.9978e-04
Validation prc-auc = 0.242615
Epoch 70
Validation prc-auc = 0.242499
Epoch 71
Validation prc-auc = 0.254930
Epoch 72
Validation prc-auc = 0.275424
Epoch 73
Validation prc-auc = 0.268193
Epoch 74
Loss = 2.9544e-02, PNorm = 84.1494, GNorm = 0.5043, lr_0 = 8.9209e-04
Validation prc-auc = 0.267116
Epoch 75
Validation prc-auc = 0.281949
Epoch 76
Validation prc-auc = 0.276047
Epoch 77
Validation prc-auc = 0.205832
Epoch 78
Validation prc-auc = 0.183349
Epoch 79
Loss = 7.6917e-02, PNorm = 84.4472, GNorm = 2.5060, lr_0 = 8.8447e-04
Validation prc-auc = 0.253312
Epoch 80
Validation prc-auc = 0.269865
Epoch 81
Validation prc-auc = 0.248978
Epoch 82
Validation prc-auc = 0.212942
Epoch 83
Validation prc-auc = 0.272852
Epoch 84
Loss = 5.6095e-02, PNorm = 84.8400, GNorm = 1.7676, lr_0 = 8.7691e-04
Validation prc-auc = 0.270776
Epoch 85
Validation prc-auc = 0.279112
Epoch 86
Validation prc-auc = 0.261560
Epoch 87
Validation prc-auc = 0.245093
Epoch 88
Validation prc-auc = 0.255377
Epoch 89
Loss = 2.8817e-02, PNorm = 85.1837, GNorm = 0.2151, lr_0 = 8.6941e-04
Validation prc-auc = 0.269885
Epoch 90
Validation prc-auc = 0.274003
Epoch 91
Validation prc-auc = 0.263056
Epoch 92
Validation prc-auc = 0.258085
Epoch 93
Validation prc-auc = 0.258102
Epoch 94
Loss = 1.4169e-02, PNorm = 85.4474, GNorm = 0.4876, lr_0 = 8.6198e-04
Validation prc-auc = 0.253735
Epoch 95
Validation prc-auc = 0.248421
Epoch 96
Validation prc-auc = 0.236368
Epoch 97
Validation prc-auc = 0.212348
Epoch 98
Validation prc-auc = 0.202274
Epoch 99
Loss = 1.1080e-02, PNorm = 85.6194, GNorm = 0.1793, lr_0 = 8.5461e-04
Validation prc-auc = 0.210346
Epoch 100
Validation prc-auc = 0.226382
Epoch 101
Validation prc-auc = 0.233868
Epoch 102
Validation prc-auc = 0.234147
Epoch 103
Validation prc-auc = 0.232578
Epoch 104
Loss = 5.8039e-03, PNorm = 85.7685, GNorm = 0.4751, lr_0 = 8.4730e-04
Validation prc-auc = 0.234665
Epoch 105
Validation prc-auc = 0.245773
Epoch 106
Validation prc-auc = 0.229765
Epoch 107
Validation prc-auc = 0.202668
Epoch 108
Validation prc-auc = 0.198487
Epoch 109
Loss = 3.3261e-03, PNorm = 85.8822, GNorm = 0.0878, lr_0 = 8.4006e-04
Validation prc-auc = 0.197556
Epoch 110
Validation prc-auc = 0.192250
Epoch 111
Validation prc-auc = 0.195907
Epoch 112
Validation prc-auc = 0.221345
Epoch 113
Validation prc-auc = 0.256502
Epoch 114
Loss = 4.8304e-03, PNorm = 85.9815, GNorm = 0.1888, lr_0 = 8.3288e-04
Validation prc-auc = 0.288530
Epoch 115
Validation prc-auc = 0.296416
Epoch 116
Validation prc-auc = 0.291004
Epoch 117
Validation prc-auc = 0.292835
Epoch 118
Validation prc-auc = 0.258954
Epoch 119
Loss = 1.6431e-03, PNorm = 86.0798, GNorm = 0.0257, lr_0 = 8.2576e-04
Validation prc-auc = 0.247764
Epoch 120
Validation prc-auc = 0.210904
Epoch 121
Validation prc-auc = 0.208080
Epoch 122
Validation prc-auc = 0.206742
Epoch 123
Validation prc-auc = 0.206940
Epoch 124
Loss = 9.5905e-04, PNorm = 86.1539, GNorm = 0.0384, lr_0 = 8.1870e-04
Validation prc-auc = 0.203504
Epoch 125
Validation prc-auc = 0.204715
Epoch 126
Validation prc-auc = 0.211334
Epoch 127
Validation prc-auc = 0.214867
Epoch 128
Validation prc-auc = 0.215314
Epoch 129
Loss = 4.8367e-04, PNorm = 86.2101, GNorm = 0.0060, lr_0 = 8.1170e-04
Validation prc-auc = 0.215197
Epoch 130
Validation prc-auc = 0.222110
Epoch 131
Validation prc-auc = 0.222112
Epoch 132
Validation prc-auc = 0.229797
Epoch 133
Validation prc-auc = 0.215422
Epoch 134
Loss = 4.2070e-03, PNorm = 86.2455, GNorm = 0.3350, lr_0 = 8.0476e-04
Validation prc-auc = 0.214897
Epoch 135
Validation prc-auc = 0.229929
Epoch 136
Validation prc-auc = 0.239647
Epoch 137
Validation prc-auc = 0.246353
Epoch 138
Validation prc-auc = 0.250734
Epoch 139
Loss = 6.0420e-04, PNorm = 86.3173, GNorm = 0.0343, lr_0 = 7.9788e-04
Validation prc-auc = 0.250647
Epoch 140
Validation prc-auc = 0.257317
Epoch 141
Validation prc-auc = 0.248711
Epoch 142
Validation prc-auc = 0.235555
Epoch 143
Validation prc-auc = 0.233822
Epoch 144
Loss = 6.8669e-04, PNorm = 86.3478, GNorm = 0.0313, lr_0 = 7.9106e-04
Validation prc-auc = 0.212841
Epoch 145
Validation prc-auc = 0.205824
Epoch 146
Validation prc-auc = 0.203542
Epoch 147
Validation prc-auc = 0.202880
Epoch 148
Validation prc-auc = 0.201224
Epoch 149
Loss = 4.4334e-04, PNorm = 86.4050, GNorm = 0.0098, lr_0 = 7.8430e-04
Validation prc-auc = 0.209669
Epoch 150
Validation prc-auc = 0.204278
Epoch 151
Validation prc-auc = 0.207228
Epoch 152
Validation prc-auc = 0.207041
Epoch 153
Validation prc-auc = 0.207041
Epoch 154
Loss = 3.5738e-04, PNorm = 86.4404, GNorm = 0.0020, lr_0 = 7.7759e-04
Validation prc-auc = 0.212598
Epoch 155
Validation prc-auc = 0.220758
Epoch 156
Validation prc-auc = 0.249183
Epoch 157
Validation prc-auc = 0.249469
Epoch 158
Validation prc-auc = 0.250900
Epoch 159
Loss = 9.2051e-04, PNorm = 86.4616, GNorm = 0.0203, lr_0 = 7.7095e-04
Validation prc-auc = 0.251085
Epoch 160
Validation prc-auc = 0.250245
Epoch 161
Validation prc-auc = 0.250634
Epoch 162
Validation prc-auc = 0.246110
Epoch 163
Validation prc-auc = 0.246093
Epoch 164
Loss = 3.2442e-04, PNorm = 86.4779, GNorm = 0.0125, lr_0 = 7.6436e-04
Validation prc-auc = 0.246118
Epoch 165
Validation prc-auc = 0.245354
Epoch 166
Validation prc-auc = 0.245917
Epoch 167
Validation prc-auc = 0.245151
Epoch 168
Validation prc-auc = 0.245481
Epoch 169
Loss = 1.9056e-04, PNorm = 86.4903, GNorm = 0.0037, lr_0 = 7.5782e-04
Validation prc-auc = 0.243320
Epoch 170
Validation prc-auc = 0.243397
Epoch 171
Validation prc-auc = 0.243613
Epoch 172
Validation prc-auc = 0.243942
Epoch 173
Validation prc-auc = 0.244307
Epoch 174
Loss = 4.2736e-04, PNorm = 86.5008, GNorm = 0.0080, lr_0 = 7.5134e-04
Validation prc-auc = 0.244400
Epoch 175
Validation prc-auc = 0.244494
Epoch 176
Validation prc-auc = 0.244454
Epoch 177
Validation prc-auc = 0.244431
Epoch 178
Validation prc-auc = 0.243393
Epoch 179
Loss = 2.6818e-04, PNorm = 86.5116, GNorm = 0.0115, lr_0 = 7.4492e-04
Validation prc-auc = 0.243267
Epoch 180
Validation prc-auc = 0.238144
Epoch 181
Validation prc-auc = 0.235473
Epoch 182
Validation prc-auc = 0.235404
Epoch 183
Validation prc-auc = 0.235608
Epoch 184
Loss = 1.8863e-04, PNorm = 86.5207, GNorm = 0.0080, lr_0 = 7.3855e-04
Validation prc-auc = 0.235624
Epoch 185
Validation prc-auc = 0.235521
Epoch 186
Validation prc-auc = 0.240956
Epoch 187
Validation prc-auc = 0.240956
Epoch 188
Validation prc-auc = 0.241229
Epoch 189
Loss = 2.0757e-04, PNorm = 86.5293, GNorm = 0.0066, lr_0 = 7.3224e-04
Validation prc-auc = 0.241229
Epoch 190
Validation prc-auc = 0.241110
Epoch 191
Validation prc-auc = 0.241110
Epoch 192
Validation prc-auc = 0.241110
Epoch 193
Validation prc-auc = 0.241021
Epoch 194
Loss = 6.9810e-05, PNorm = 86.5377, GNorm = 0.0019, lr_0 = 7.2598e-04
Validation prc-auc = 0.241021
Epoch 195
Validation prc-auc = 0.241140
Epoch 196
Validation prc-auc = 0.241040
Epoch 197
Validation prc-auc = 0.240974
Epoch 198
Validation prc-auc = 0.241132
Epoch 199
Loss = 6.9200e-05, PNorm = 86.5450, GNorm = 0.0011, lr_0 = 7.1977e-04
Validation prc-auc = 0.241248
Epoch 200
Validation prc-auc = 0.241248
Epoch 201
Validation prc-auc = 0.241436
Epoch 202
Validation prc-auc = 0.241436
Epoch 203
Validation prc-auc = 0.241436
Epoch 204
Loss = 1.2017e-04, PNorm = 86.5503, GNorm = 0.0028, lr_0 = 7.1362e-04
Validation prc-auc = 0.241436
Epoch 205
Validation prc-auc = 0.241436
Epoch 206
Validation prc-auc = 0.244757
Epoch 207
Validation prc-auc = 0.244883
Epoch 208
Validation prc-auc = 0.244796
Epoch 209
Loss = 6.0873e-05, PNorm = 86.5555, GNorm = 0.0016, lr_0 = 7.0752e-04
Validation prc-auc = 0.241373
Epoch 210
Validation prc-auc = 0.242334
Epoch 211
Validation prc-auc = 0.242443
Epoch 212
Validation prc-auc = 0.242608
Epoch 213
Validation prc-auc = 0.242856
Epoch 214
Loss = 1.3249e-04, PNorm = 86.5609, GNorm = 0.0044, lr_0 = 7.0147e-04
Validation prc-auc = 0.242639
Epoch 215
Validation prc-auc = 0.242574
Epoch 216
Validation prc-auc = 0.242574
Epoch 217
Validation prc-auc = 0.242574
Epoch 218
Validation prc-auc = 0.245897
Epoch 219
Loss = 5.6057e-05, PNorm = 86.5676, GNorm = 0.0017, lr_0 = 6.9548e-04
Validation prc-auc = 0.245913
Epoch 220
Validation prc-auc = 0.249503
Epoch 221
Validation prc-auc = 0.249583
Epoch 222
Validation prc-auc = 0.238346
Epoch 223
Validation prc-auc = 0.238017
Epoch 224
Loss = 7.1717e-05, PNorm = 86.5741, GNorm = 0.0008, lr_0 = 6.8953e-04
Validation prc-auc = 0.237636
Epoch 225
Validation prc-auc = 0.237557
Epoch 226
Validation prc-auc = 0.237557
Epoch 227
Validation prc-auc = 0.237557
Epoch 228
Validation prc-auc = 0.242161
Epoch 229
Loss = 7.9222e-05, PNorm = 86.5786, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.242221
Epoch 230
Validation prc-auc = 0.238465
Epoch 231
Validation prc-auc = 0.242052
Epoch 232
Validation prc-auc = 0.238603
Epoch 233
Validation prc-auc = 0.241255
Epoch 234
Loss = 4.5943e-05, PNorm = 86.5829, GNorm = 0.0010, lr_0 = 6.7779e-04
Validation prc-auc = 0.241878
Epoch 235
Validation prc-auc = 0.245466
Epoch 236
Validation prc-auc = 0.246428
Epoch 237
Validation prc-auc = 0.240818
Epoch 238
Validation prc-auc = 0.237349
Epoch 239
Loss = 3.8840e-05, PNorm = 86.5877, GNorm = 0.0011, lr_0 = 6.7200e-04
Validation prc-auc = 0.241673
Epoch 240
Validation prc-auc = 0.241791
Epoch 241
Validation prc-auc = 0.242033
Epoch 242
Validation prc-auc = 0.248715
Epoch 243
Validation prc-auc = 0.248478
Epoch 244
Loss = 3.0307e-05, PNorm = 86.5913, GNorm = 0.0014, lr_0 = 6.6625e-04
Validation prc-auc = 0.247657
Epoch 245
Validation prc-auc = 0.258528
Epoch 246
Validation prc-auc = 0.247285
Epoch 247
Validation prc-auc = 0.245310
Epoch 248
Validation prc-auc = 0.245393
Epoch 249
Loss = 2.7917e-05, PNorm = 86.5958, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.239596
Epoch 250
Validation prc-auc = 0.245767
Epoch 251
Validation prc-auc = 0.245592
Epoch 252
Validation prc-auc = 0.245608
Epoch 253
Validation prc-auc = 0.251031
Epoch 254
Loss = 3.1287e-05, PNorm = 86.6002, GNorm = 0.0007, lr_0 = 6.5491e-04
Validation prc-auc = 0.251048
Epoch 255
Validation prc-auc = 0.250844
Epoch 256
Validation prc-auc = 0.254301
Epoch 257
Validation prc-auc = 0.257112
Epoch 258
Validation prc-auc = 0.256776
Epoch 259
Loss = 4.4208e-05, PNorm = 86.6044, GNorm = 0.0050, lr_0 = 6.4931e-04
Validation prc-auc = 0.255416
Epoch 260
Validation prc-auc = 0.250021
Epoch 261
Validation prc-auc = 0.248634
Epoch 262
Validation prc-auc = 0.245542
Epoch 263
Validation prc-auc = 0.245542
Epoch 264
Loss = 3.8877e-05, PNorm = 86.6086, GNorm = 0.0003, lr_0 = 6.4376e-04
Validation prc-auc = 0.245999
Epoch 265
Validation prc-auc = 0.245936
Epoch 266
Validation prc-auc = 0.247678
Epoch 267
Validation prc-auc = 0.251398
Epoch 268
Validation prc-auc = 0.269290
Epoch 269
Loss = 2.9516e-05, PNorm = 86.6109, GNorm = 0.0013, lr_0 = 6.3826e-04
Validation prc-auc = 0.269625
Epoch 270
Validation prc-auc = 0.269625
Epoch 271
Validation prc-auc = 0.269516
Epoch 272
Validation prc-auc = 0.266489
Epoch 273
Validation prc-auc = 0.255238
Epoch 274
Loss = 2.3526e-05, PNorm = 86.6148, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.248307
Epoch 275
Validation prc-auc = 0.248124
Epoch 276
Validation prc-auc = 0.248124
Epoch 277
Validation prc-auc = 0.247711
Epoch 278
Validation prc-auc = 0.247905
Epoch 279
Loss = 1.7578e-05, PNorm = 86.6185, GNorm = 0.0015, lr_0 = 6.2739e-04
Validation prc-auc = 0.247873
Epoch 280
Validation prc-auc = 0.247758
Epoch 281
Validation prc-auc = 0.247821
Epoch 282
Validation prc-auc = 0.247629
Epoch 283
Validation prc-auc = 0.247742
Epoch 284
Loss = 2.6666e-05, PNorm = 86.6215, GNorm = 0.0014, lr_0 = 6.2203e-04
Validation prc-auc = 0.255118
Epoch 285
Validation prc-auc = 0.256661
Epoch 286
Validation prc-auc = 0.256779
Epoch 287
Validation prc-auc = 0.259250
Epoch 288
Validation prc-auc = 0.259301
Epoch 289
Loss = 1.7281e-05, PNorm = 86.6252, GNorm = 0.0011, lr_0 = 6.1671e-04
Validation prc-auc = 0.256731
Epoch 290
Validation prc-auc = 0.255539
Epoch 291
Validation prc-auc = 0.255066
Epoch 292
Validation prc-auc = 0.254954
Epoch 293
Validation prc-auc = 0.255534
Epoch 294
Loss = 4.2097e-05, PNorm = 86.6282, GNorm = 0.0024, lr_0 = 6.1144e-04
Validation prc-auc = 0.255490
Epoch 295
Validation prc-auc = 0.255597
Epoch 296
Validation prc-auc = 0.253429
Epoch 297
Validation prc-auc = 0.250955
Epoch 298
Validation prc-auc = 0.249664
Epoch 299
Loss = 2.6413e-05, PNorm = 86.6322, GNorm = 0.0020, lr_0 = 6.0621e-04
Validation prc-auc = 0.247300
Model 0 best validation prc-auc = 0.497692 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.401629
Ensemble test prc-auc = 0.401629
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.402364
	Seed 1 ==> test prc-auc = 0.387643
	Seed 2 ==> test prc-auc = 0.473718
	Seed 3 ==> test prc-auc = 0.528662
	Seed 4 ==> test prc-auc = 0.426883
	Seed 5 ==> test prc-auc = 0.387804
	Seed 6 ==> test prc-auc = 0.381926
	Seed 7 ==> test prc-auc = 0.512151
	Seed 8 ==> test prc-auc = 0.437878
	Seed 9 ==> test prc-auc = 0.401629
Overall test prc-auc = 0.434066 +/- 0.050750
Elapsed time = 0:08:46
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.05,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_prc-auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.445707
Epoch 1
Validation prc-auc = 0.458308
Epoch 2
Validation prc-auc = 0.486724
Epoch 3
Validation prc-auc = 0.446652
Epoch 4
Loss = 5.9146e-01, PNorm = 78.5516, GNorm = 6.0189, lr_0 = 6.5000e-04
Validation prc-auc = 0.423135
Epoch 5
Validation prc-auc = 0.353443
Epoch 6
Validation prc-auc = 0.357471
Epoch 7
Validation prc-auc = 0.330555
Epoch 8
Validation prc-auc = 0.281123
Epoch 9
Loss = 7.6735e-01, PNorm = 78.7998, GNorm = 20.4609, lr_0 = 9.9743e-04
Validation prc-auc = 0.403514
Epoch 10
Validation prc-auc = 0.444280
Epoch 11
Validation prc-auc = 0.486078
Epoch 12
Validation prc-auc = 0.437510
Epoch 13
Validation prc-auc = 0.431015
Epoch 14
Loss = 5.1773e-01, PNorm = 79.1107, GNorm = 1.8584, lr_0 = 9.8890e-04
Validation prc-auc = 0.330359
Epoch 15
Validation prc-auc = 0.417203
Epoch 16
Validation prc-auc = 0.406734
Epoch 17
Validation prc-auc = 0.411512
Epoch 18
Validation prc-auc = 0.442178
Epoch 19
Loss = 5.3559e-01, PNorm = 79.4173, GNorm = 5.7605, lr_0 = 9.8045e-04
Validation prc-auc = 0.451489
Epoch 20
Validation prc-auc = 0.271484
Epoch 21
Validation prc-auc = 0.438021
Epoch 22
Validation prc-auc = 0.309709
Epoch 23
Validation prc-auc = 0.164991
Epoch 24
Loss = 5.1603e-01, PNorm = 79.7061, GNorm = 3.9063, lr_0 = 9.7207e-04
Validation prc-auc = 0.326223
Epoch 25
Validation prc-auc = 0.357101
Epoch 26
Validation prc-auc = 0.349736
Epoch 27
Validation prc-auc = 0.350833
Epoch 28
Validation prc-auc = 0.347300
Epoch 29
Loss = 4.4670e-01, PNorm = 80.0649, GNorm = 5.2925, lr_0 = 9.6376e-04
Validation prc-auc = 0.339963
Epoch 30
Validation prc-auc = 0.361679
Epoch 31
Validation prc-auc = 0.340161
Epoch 32
Validation prc-auc = 0.346327
Epoch 33
Validation prc-auc = 0.357100
Epoch 34
Loss = 3.6524e-01, PNorm = 80.4179, GNorm = 1.7099, lr_0 = 9.5552e-04
Validation prc-auc = 0.340565
Epoch 35
Validation prc-auc = 0.333382
Epoch 36
Validation prc-auc = 0.303599
Epoch 37
Validation prc-auc = 0.275057
Epoch 38
Validation prc-auc = 0.351124
Epoch 39
Loss = 3.5502e-01, PNorm = 80.7368, GNorm = 3.2371, lr_0 = 9.4735e-04
Validation prc-auc = 0.341097
Epoch 40
Validation prc-auc = 0.439518
Epoch 41
Validation prc-auc = 0.368610
Epoch 42
Validation prc-auc = 0.367481
Epoch 43
Validation prc-auc = 0.225364
Epoch 44
Loss = 2.5396e-01, PNorm = 81.1163, GNorm = 0.8848, lr_0 = 9.3925e-04
Validation prc-auc = 0.315748
Epoch 45
Validation prc-auc = 0.330290
Epoch 46
Validation prc-auc = 0.349946
Epoch 47
Validation prc-auc = 0.329768
Epoch 48
Validation prc-auc = 0.347256
Epoch 49
Loss = 2.4885e-01, PNorm = 81.5223, GNorm = 4.5164, lr_0 = 9.3122e-04
Validation prc-auc = 0.308439
Epoch 50
Validation prc-auc = 0.173983
Epoch 51
Validation prc-auc = 0.259395
Epoch 52
Validation prc-auc = 0.338578
Epoch 53
Validation prc-auc = 0.360054
Epoch 54
Loss = 2.4122e-01, PNorm = 81.9848, GNorm = 1.9906, lr_0 = 9.2326e-04
Validation prc-auc = 0.347880
Epoch 55
Validation prc-auc = 0.283884
Epoch 56
Validation prc-auc = 0.305573
Epoch 57
Validation prc-auc = 0.313327
Epoch 58
Validation prc-auc = 0.337140
Epoch 59
Loss = 1.5524e-01, PNorm = 82.4705, GNorm = 3.7498, lr_0 = 9.1537e-04
Validation prc-auc = 0.340310
Epoch 60
Validation prc-auc = 0.320154
Epoch 61
Validation prc-auc = 0.354776
Epoch 62
Validation prc-auc = 0.367880
Epoch 63
Validation prc-auc = 0.358730
Epoch 64
Loss = 7.5256e-02, PNorm = 82.8959, GNorm = 0.8970, lr_0 = 9.0754e-04
Validation prc-auc = 0.372602
Epoch 65
Validation prc-auc = 0.358680
Epoch 66
Validation prc-auc = 0.293166
Epoch 67
Validation prc-auc = 0.219673
Epoch 68
Validation prc-auc = 0.296784
Epoch 69
Loss = 7.5535e-02, PNorm = 83.2407, GNorm = 1.3195, lr_0 = 8.9978e-04
Validation prc-auc = 0.295494
Epoch 70
Validation prc-auc = 0.274386
Epoch 71
Validation prc-auc = 0.297512
Epoch 72
Validation prc-auc = 0.306965
Epoch 73
Validation prc-auc = 0.283575
Epoch 74
Loss = 9.9013e-02, PNorm = 83.6169, GNorm = 2.4787, lr_0 = 8.9209e-04
Validation prc-auc = 0.239506
Epoch 75
Validation prc-auc = 0.271515
Epoch 76
Validation prc-auc = 0.263982
Epoch 77
Validation prc-auc = 0.202906
Epoch 78
Validation prc-auc = 0.298382
Epoch 79
Loss = 1.0069e-01, PNorm = 84.0248, GNorm = 1.3521, lr_0 = 8.8447e-04
Validation prc-auc = 0.336569
Epoch 80
Validation prc-auc = 0.279503
Epoch 81
Validation prc-auc = 0.299879
Epoch 82
Validation prc-auc = 0.345763
Epoch 83
Validation prc-auc = 0.325699
Epoch 84
Loss = 8.1490e-02, PNorm = 84.4339, GNorm = 2.0025, lr_0 = 8.7691e-04
Validation prc-auc = 0.271849
Epoch 85
Validation prc-auc = 0.245776
Epoch 86
Validation prc-auc = 0.263068
Epoch 87
Validation prc-auc = 0.301349
Epoch 88
Validation prc-auc = 0.292092
Epoch 89
Loss = 5.0683e-02, PNorm = 84.8066, GNorm = 0.3284, lr_0 = 8.6941e-04
Validation prc-auc = 0.284103
Epoch 90
Validation prc-auc = 0.314465
Epoch 91
Validation prc-auc = 0.327746
Epoch 92
Validation prc-auc = 0.329459
Epoch 93
Validation prc-auc = 0.303492
Epoch 94
Loss = 5.3445e-02, PNorm = 85.0653, GNorm = 0.3499, lr_0 = 8.6198e-04
Validation prc-auc = 0.277749
Epoch 95
Validation prc-auc = 0.256772
Epoch 96
Validation prc-auc = 0.278746
Epoch 97
Validation prc-auc = 0.268610
Epoch 98
Validation prc-auc = 0.264428
Epoch 99
Loss = 8.9455e-03, PNorm = 85.3073, GNorm = 0.1759, lr_0 = 8.5461e-04
Validation prc-auc = 0.270707
Epoch 100
Validation prc-auc = 0.264025
Epoch 101
Validation prc-auc = 0.272788
Epoch 102
Validation prc-auc = 0.274327
Epoch 103
Validation prc-auc = 0.266981
Epoch 104
Loss = 2.8928e-02, PNorm = 85.4931, GNorm = 0.9130, lr_0 = 8.4730e-04
Validation prc-auc = 0.258173
Epoch 105
Validation prc-auc = 0.246047
Epoch 106
Validation prc-auc = 0.245920
Epoch 107
Validation prc-auc = 0.270633
Epoch 108
Validation prc-auc = 0.285974
Epoch 109
Loss = 1.3276e-02, PNorm = 85.6793, GNorm = 0.1830, lr_0 = 8.4006e-04
Validation prc-auc = 0.284292
Epoch 110
Validation prc-auc = 0.291069
Epoch 111
Validation prc-auc = 0.279525
Epoch 112
Validation prc-auc = 0.277428
Epoch 113
Validation prc-auc = 0.279185
Epoch 114
Loss = 1.8808e-02, PNorm = 85.8312, GNorm = 0.0561, lr_0 = 8.3288e-04
Validation prc-auc = 0.276110
Epoch 115
Validation prc-auc = 0.271988
Epoch 116
Validation prc-auc = 0.271654
Epoch 117
Validation prc-auc = 0.270823
Epoch 118
Validation prc-auc = 0.270737
Epoch 119
Loss = 4.4386e-03, PNorm = 85.9360, GNorm = 0.0552, lr_0 = 8.2576e-04
Validation prc-auc = 0.265659
Epoch 120
Validation prc-auc = 0.268518
Epoch 121
Validation prc-auc = 0.263683
Epoch 122
Validation prc-auc = 0.250535
Epoch 123
Validation prc-auc = 0.229241
Epoch 124
Loss = 7.8241e-03, PNorm = 86.0179, GNorm = 0.0496, lr_0 = 8.1870e-04
Validation prc-auc = 0.205385
Epoch 125
Validation prc-auc = 0.239006
Epoch 126
Validation prc-auc = 0.258098
Epoch 127
Validation prc-auc = 0.257373
Epoch 128
Validation prc-auc = 0.263149
Epoch 129
Loss = 2.7940e-02, PNorm = 86.1505, GNorm = 0.4554, lr_0 = 8.1170e-04
Validation prc-auc = 0.271304
Epoch 130
Validation prc-auc = 0.274956
Epoch 131
Validation prc-auc = 0.271552
Epoch 132
Validation prc-auc = 0.268919
Epoch 133
Validation prc-auc = 0.270483
Epoch 134
Loss = 3.8735e-03, PNorm = 86.2603, GNorm = 0.2637, lr_0 = 8.0476e-04
Validation prc-auc = 0.276090
Epoch 135
Validation prc-auc = 0.286859
Epoch 136
Validation prc-auc = 0.281980
Epoch 137
Validation prc-auc = 0.280220
Epoch 138
Validation prc-auc = 0.269243
Epoch 139
Loss = 1.9388e-03, PNorm = 86.3415, GNorm = 0.0418, lr_0 = 7.9788e-04
Validation prc-auc = 0.268408
Epoch 140
Validation prc-auc = 0.256887
Epoch 141
Validation prc-auc = 0.260881
Epoch 142
Validation prc-auc = 0.244569
Epoch 143
Validation prc-auc = 0.241979
Epoch 144
Loss = 1.3388e-03, PNorm = 86.4374, GNorm = 0.0374, lr_0 = 7.9106e-04
Validation prc-auc = 0.231179
Epoch 145
Validation prc-auc = 0.231124
Epoch 146
Validation prc-auc = 0.244250
Epoch 147
Validation prc-auc = 0.286062
Epoch 148
Validation prc-auc = 0.272986
Epoch 149
Loss = 4.9021e-02, PNorm = 86.5866, GNorm = 2.7082, lr_0 = 7.8430e-04
Validation prc-auc = 0.232712
Epoch 150
Validation prc-auc = 0.238059
Epoch 151
Validation prc-auc = 0.248517
Epoch 152
Validation prc-auc = 0.224486
Epoch 153
Validation prc-auc = 0.212868
Epoch 154
Loss = 7.0229e-02, PNorm = 86.9283, GNorm = 0.7214, lr_0 = 7.7759e-04
Validation prc-auc = 0.217212
Epoch 155
Validation prc-auc = 0.272185
Epoch 156
Validation prc-auc = 0.293776
Epoch 157
Validation prc-auc = 0.268372
Epoch 158
Validation prc-auc = 0.227642
Epoch 159
Loss = 1.3424e-02, PNorm = 87.2404, GNorm = 0.0886, lr_0 = 7.7095e-04
Validation prc-auc = 0.232850
Epoch 160
Validation prc-auc = 0.238356
Epoch 161
Validation prc-auc = 0.235740
Epoch 162
Validation prc-auc = 0.236910
Epoch 163
Validation prc-auc = 0.235553
Epoch 164
Loss = 2.3523e-02, PNorm = 87.4460, GNorm = 0.3470, lr_0 = 7.6436e-04
Validation prc-auc = 0.231336
Epoch 165
Validation prc-auc = 0.221248
Epoch 166
Validation prc-auc = 0.219767
Epoch 167
Validation prc-auc = 0.221980
Epoch 168
Validation prc-auc = 0.219601
Epoch 169
Loss = 1.6038e-03, PNorm = 87.6006, GNorm = 0.0283, lr_0 = 7.5782e-04
Validation prc-auc = 0.229195
Epoch 170
Validation prc-auc = 0.241542
Epoch 171
Validation prc-auc = 0.245334
Epoch 172
Validation prc-auc = 0.253989
Epoch 173
Validation prc-auc = 0.266207
Epoch 174
Loss = 5.2102e-03, PNorm = 87.6887, GNorm = 0.0197, lr_0 = 7.5134e-04
Validation prc-auc = 0.264828
Epoch 175
Validation prc-auc = 0.264145
Epoch 176
Validation prc-auc = 0.271491
Epoch 177
Validation prc-auc = 0.276224
Epoch 178
Validation prc-auc = 0.275161
Epoch 179
Loss = 1.3393e-03, PNorm = 87.7670, GNorm = 0.0348, lr_0 = 7.4492e-04
Validation prc-auc = 0.274605
Epoch 180
Validation prc-auc = 0.277417
Epoch 181
Validation prc-auc = 0.294547
Epoch 182
Validation prc-auc = 0.294507
Epoch 183
Validation prc-auc = 0.301067
Epoch 184
Loss = 6.7950e-04, PNorm = 87.8157, GNorm = 0.0139, lr_0 = 7.3855e-04
Validation prc-auc = 0.301259
Epoch 185
Validation prc-auc = 0.303262
Epoch 186
Validation prc-auc = 0.286288
Epoch 187
Validation prc-auc = 0.286300
Epoch 188
Validation prc-auc = 0.295979
Epoch 189
Loss = 9.6690e-04, PNorm = 87.8530, GNorm = 0.0188, lr_0 = 7.3224e-04
Validation prc-auc = 0.301419
Epoch 190
Validation prc-auc = 0.300685
Epoch 191
Validation prc-auc = 0.294702
Epoch 192
Validation prc-auc = 0.282978
Epoch 193
Validation prc-auc = 0.270727
Epoch 194
Loss = 1.2198e-03, PNorm = 87.8885, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.266163
Epoch 195
Validation prc-auc = 0.270245
Epoch 196
Validation prc-auc = 0.270173
Epoch 197
Validation prc-auc = 0.265934
Epoch 198
Validation prc-auc = 0.270732
Epoch 199
Loss = 6.6885e-04, PNorm = 87.9168, GNorm = 0.0120, lr_0 = 7.1977e-04
Validation prc-auc = 0.271415
Epoch 200
Validation prc-auc = 0.288482
Epoch 201
Validation prc-auc = 0.284893
Epoch 202
Validation prc-auc = 0.285060
Epoch 203
Validation prc-auc = 0.307681
Epoch 204
Loss = 8.4359e-05, PNorm = 87.9388, GNorm = 0.0025, lr_0 = 7.1362e-04
Validation prc-auc = 0.314679
Epoch 205
Validation prc-auc = 0.305623
Epoch 206
Validation prc-auc = 0.310165
Epoch 207
Validation prc-auc = 0.310124
Epoch 208
Validation prc-auc = 0.301500
Epoch 209
Loss = 5.2673e-04, PNorm = 87.9587, GNorm = 0.0489, lr_0 = 7.0752e-04
Validation prc-auc = 0.300496
Epoch 210
Validation prc-auc = 0.284767
Epoch 211
Validation prc-auc = 0.279483
Epoch 212
Validation prc-auc = 0.279546
Epoch 213
Validation prc-auc = 0.279826
Epoch 214
Loss = 1.2627e-04, PNorm = 87.9753, GNorm = 0.0048, lr_0 = 7.0147e-04
Validation prc-auc = 0.284899
Epoch 215
Validation prc-auc = 0.289604
Epoch 216
Validation prc-auc = 0.300577
Epoch 217
Validation prc-auc = 0.300990
Epoch 218
Validation prc-auc = 0.301239
Epoch 219
Loss = 2.5233e-04, PNorm = 87.9907, GNorm = 0.0108, lr_0 = 6.9548e-04
Validation prc-auc = 0.301651
Epoch 220
Validation prc-auc = 0.301411
Epoch 221
Validation prc-auc = 0.301202
Epoch 222
Validation prc-auc = 0.301063
Epoch 223
Validation prc-auc = 0.300274
Epoch 224
Loss = 2.9315e-04, PNorm = 88.0053, GNorm = 0.0191, lr_0 = 6.8953e-04
Validation prc-auc = 0.295728
Epoch 225
Validation prc-auc = 0.295912
Epoch 226
Validation prc-auc = 0.291701
Epoch 227
Validation prc-auc = 0.288034
Epoch 228
Validation prc-auc = 0.291969
Epoch 229
Loss = 2.4461e-04, PNorm = 88.0127, GNorm = 0.0141, lr_0 = 6.8364e-04
Validation prc-auc = 0.296665
Epoch 230
Validation prc-auc = 0.306781
Epoch 231
Validation prc-auc = 0.311456
Epoch 232
Validation prc-auc = 0.311396
Epoch 233
Validation prc-auc = 0.311396
Epoch 234
Loss = 1.2914e-04, PNorm = 88.0232, GNorm = 0.0034, lr_0 = 6.7779e-04
Validation prc-auc = 0.303059
Epoch 235
Validation prc-auc = 0.302870
Epoch 236
Validation prc-auc = 0.302869
Epoch 237
Validation prc-auc = 0.302747
Epoch 238
Validation prc-auc = 0.302210
Epoch 239
Loss = 1.2278e-04, PNorm = 88.0353, GNorm = 0.0029, lr_0 = 6.7200e-04
Validation prc-auc = 0.302092
Epoch 240
Validation prc-auc = 0.302092
Epoch 241
Validation prc-auc = 0.297142
Epoch 242
Validation prc-auc = 0.293438
Epoch 243
Validation prc-auc = 0.293578
Epoch 244
Loss = 7.1495e-05, PNorm = 88.0434, GNorm = 0.0039, lr_0 = 6.6625e-04
Validation prc-auc = 0.293645
Epoch 245
Validation prc-auc = 0.297940
Epoch 246
Validation prc-auc = 0.298526
Epoch 247
Validation prc-auc = 0.298917
Epoch 248
Validation prc-auc = 0.302558
Epoch 249
Loss = 1.0654e-04, PNorm = 88.0486, GNorm = 0.0014, lr_0 = 6.6056e-04
Validation prc-auc = 0.313573
Epoch 250
Validation prc-auc = 0.313504
Epoch 251
Validation prc-auc = 0.313366
Epoch 252
Validation prc-auc = 0.313295
Epoch 253
Validation prc-auc = 0.302538
Epoch 254
Loss = 8.0787e-05, PNorm = 88.0568, GNorm = 0.0034, lr_0 = 6.5491e-04
Validation prc-auc = 0.298417
Epoch 255
Validation prc-auc = 0.298276
Epoch 256
Validation prc-auc = 0.297689
Epoch 257
Validation prc-auc = 0.297427
Epoch 258
Validation prc-auc = 0.297499
Epoch 259
Loss = 5.9110e-05, PNorm = 88.0638, GNorm = 0.0015, lr_0 = 6.4931e-04
Validation prc-auc = 0.298014
Epoch 260
Validation prc-auc = 0.331347
Epoch 261
Validation prc-auc = 0.331277
Epoch 262
Validation prc-auc = 0.331151
Epoch 263
Validation prc-auc = 0.331130
Epoch 264
Loss = 9.1253e-05, PNorm = 88.0694, GNorm = 0.0064, lr_0 = 6.4376e-04
Validation prc-auc = 0.335768
Epoch 265
Validation prc-auc = 0.336187
Epoch 266
Validation prc-auc = 0.335446
Epoch 267
Validation prc-auc = 0.340438
Epoch 268
Validation prc-auc = 0.340500
Epoch 269
Loss = 3.4476e-05, PNorm = 88.0747, GNorm = 0.0014, lr_0 = 6.3826e-04
Validation prc-auc = 0.340638
Epoch 270
Validation prc-auc = 0.340264
Epoch 271
Validation prc-auc = 0.336282
Epoch 272
Validation prc-auc = 0.340074
Epoch 273
Validation prc-auc = 0.340485
Epoch 274
Loss = 5.8804e-05, PNorm = 88.0807, GNorm = 0.0022, lr_0 = 6.3280e-04
Validation prc-auc = 0.346581
Epoch 275
Validation prc-auc = 0.346718
Epoch 276
Validation prc-auc = 0.356756
Epoch 277
Validation prc-auc = 0.356595
Epoch 278
Validation prc-auc = 0.356595
Epoch 279
Loss = 2.7580e-05, PNorm = 88.0861, GNorm = 0.0006, lr_0 = 6.2739e-04
Validation prc-auc = 0.356722
Epoch 280
Validation prc-auc = 0.347937
Epoch 281
Validation prc-auc = 0.347190
Epoch 282
Validation prc-auc = 0.346615
Epoch 283
Validation prc-auc = 0.346484
Epoch 284
Loss = 2.2360e-05, PNorm = 88.0910, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.346354
Epoch 285
Validation prc-auc = 0.346354
Epoch 286
Validation prc-auc = 0.340203
Epoch 287
Validation prc-auc = 0.340214
Epoch 288
Validation prc-auc = 0.335581
Epoch 289
Loss = 3.0651e-05, PNorm = 88.0939, GNorm = 0.0010, lr_0 = 6.1671e-04
Validation prc-auc = 0.335678
Epoch 290
Validation prc-auc = 0.335602
Epoch 291
Validation prc-auc = 0.335329
Epoch 292
Validation prc-auc = 0.335211
Epoch 293
Validation prc-auc = 0.335675
Epoch 294
Loss = 3.2700e-05, PNorm = 88.0977, GNorm = 0.0052, lr_0 = 6.1144e-04
Validation prc-auc = 0.335876
Epoch 295
Validation prc-auc = 0.335479
Epoch 296
Validation prc-auc = 0.335553
Epoch 297
Validation prc-auc = 0.335358
Epoch 298
Validation prc-auc = 0.335749
Epoch 299
Loss = 4.0968e-05, PNorm = 88.1015, GNorm = 0.0011, lr_0 = 6.0621e-04
Validation prc-auc = 0.347635
Model 0 best validation prc-auc = 0.486724 on epoch 2
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.402364
Ensemble test prc-auc = 0.402364
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.454787
Epoch 1
Validation prc-auc = 0.455183
Epoch 2
Validation prc-auc = 0.457246
Epoch 3
Validation prc-auc = 0.460642
Epoch 4
Loss = 7.0194e-01, PNorm = 78.5504, GNorm = 3.9290, lr_0 = 6.5000e-04
Validation prc-auc = 0.442238
Epoch 5
Validation prc-auc = 0.453212
Epoch 6
Validation prc-auc = 0.457813
Epoch 7
Validation prc-auc = 0.462085
Epoch 8
Validation prc-auc = 0.472303
Epoch 9
Loss = 6.0939e-01, PNorm = 78.7958, GNorm = 9.0791, lr_0 = 9.9743e-04
Validation prc-auc = 0.462185
Epoch 10
Validation prc-auc = 0.334396
Epoch 11
Validation prc-auc = 0.306692
Epoch 12
Validation prc-auc = 0.346881
Epoch 13
Validation prc-auc = 0.310795
Epoch 14
Loss = 5.8294e-01, PNorm = 79.0813, GNorm = 2.1918, lr_0 = 9.8890e-04
Validation prc-auc = 0.269379
Epoch 15
Validation prc-auc = 0.337045
Epoch 16
Validation prc-auc = 0.434443
Epoch 17
Validation prc-auc = 0.441915
Epoch 18
Validation prc-auc = 0.418510
Epoch 19
Loss = 4.2754e-01, PNorm = 79.3729, GNorm = 1.9741, lr_0 = 9.8045e-04
Validation prc-auc = 0.392791
Epoch 20
Validation prc-auc = 0.420089
Epoch 21
Validation prc-auc = 0.403046
Epoch 22
Validation prc-auc = 0.367614
Epoch 23
Validation prc-auc = 0.337152
Epoch 24
Loss = 4.9482e-01, PNorm = 79.6858, GNorm = 6.3413, lr_0 = 9.7207e-04
Validation prc-auc = 0.267790
Epoch 25
Validation prc-auc = 0.271424
Epoch 26
Validation prc-auc = 0.276685
Epoch 27
Validation prc-auc = 0.296748
Epoch 28
Validation prc-auc = 0.338195
Epoch 29
Loss = 3.4506e-01, PNorm = 80.0312, GNorm = 0.9223, lr_0 = 9.6376e-04
Validation prc-auc = 0.416699
Epoch 30
Validation prc-auc = 0.457941
Epoch 31
Validation prc-auc = 0.450469
Epoch 32
Validation prc-auc = 0.270926
Epoch 33
Validation prc-auc = 0.334869
Epoch 34
Loss = 2.5451e-01, PNorm = 80.3966, GNorm = 2.3773, lr_0 = 9.5552e-04
Validation prc-auc = 0.389059
Epoch 35
Validation prc-auc = 0.405813
Epoch 36
Validation prc-auc = 0.377166
Epoch 37
Validation prc-auc = 0.340674
Epoch 38
Validation prc-auc = 0.234162
Epoch 39
Loss = 2.9442e-01, PNorm = 80.7763, GNorm = 2.9394, lr_0 = 9.4735e-04
Validation prc-auc = 0.199282
Epoch 40
Validation prc-auc = 0.199382
Epoch 41
Validation prc-auc = 0.341877
Epoch 42
Validation prc-auc = 0.327700
Epoch 43
Validation prc-auc = 0.348898
Epoch 44
Loss = 1.9134e-01, PNorm = 81.2134, GNorm = 4.3996, lr_0 = 9.3925e-04
Validation prc-auc = 0.336673
Epoch 45
Validation prc-auc = 0.316691
Epoch 46
Validation prc-auc = 0.336910
Epoch 47
Validation prc-auc = 0.262107
Epoch 48
Validation prc-auc = 0.213369
Epoch 49
Loss = 1.0014e-01, PNorm = 81.6307, GNorm = 1.4613, lr_0 = 9.3122e-04
Validation prc-auc = 0.276573
Epoch 50
Validation prc-auc = 0.293355
Epoch 51
Validation prc-auc = 0.294258
Epoch 52
Validation prc-auc = 0.324556
Epoch 53
Validation prc-auc = 0.287274
Epoch 54
Loss = 3.0780e-01, PNorm = 82.0069, GNorm = 3.1248, lr_0 = 9.2326e-04
Validation prc-auc = 0.275376
Epoch 55
Validation prc-auc = 0.299874
Epoch 56
Validation prc-auc = 0.317530
Epoch 57
Validation prc-auc = 0.309956
Epoch 58
Validation prc-auc = 0.262078
Epoch 59
Loss = 1.6468e-01, PNorm = 82.3623, GNorm = 0.7541, lr_0 = 9.1537e-04
Validation prc-auc = 0.258190
Epoch 60
Validation prc-auc = 0.233495
Epoch 61
Validation prc-auc = 0.216412
Epoch 62
Validation prc-auc = 0.240594
Epoch 63
Validation prc-auc = 0.211781
Epoch 64
Loss = 1.1436e-01, PNorm = 82.7665, GNorm = 2.5405, lr_0 = 9.0754e-04
Validation prc-auc = 0.201018
Epoch 65
Validation prc-auc = 0.215336
Epoch 66
Validation prc-auc = 0.193294
Epoch 67
Validation prc-auc = 0.243513
Epoch 68
Validation prc-auc = 0.351066
Epoch 69
Loss = 1.5013e-01, PNorm = 83.1409, GNorm = 0.7780, lr_0 = 8.9978e-04
Validation prc-auc = 0.197157
Epoch 70
Validation prc-auc = 0.267839
Epoch 71
Validation prc-auc = 0.256646
Epoch 72
Validation prc-auc = 0.227305
Epoch 73
Validation prc-auc = 0.221294
Epoch 74
Loss = 8.8397e-02, PNorm = 83.4788, GNorm = 0.3822, lr_0 = 8.9209e-04
Validation prc-auc = 0.243400
Epoch 75
Validation prc-auc = 0.257978
Epoch 76
Validation prc-auc = 0.249578
Epoch 77
Validation prc-auc = 0.253461
Epoch 78
Validation prc-auc = 0.267424
Epoch 79
Loss = 5.1197e-02, PNorm = 83.7954, GNorm = 0.5032, lr_0 = 8.8447e-04
Validation prc-auc = 0.286060
Epoch 80
Validation prc-auc = 0.258376
Epoch 81
Validation prc-auc = 0.256599
Epoch 82
Validation prc-auc = 0.249125
Epoch 83
Validation prc-auc = 0.232321
Epoch 84
Loss = 2.3021e-02, PNorm = 84.0485, GNorm = 0.8325, lr_0 = 8.7691e-04
Validation prc-auc = 0.240687
Epoch 85
Validation prc-auc = 0.241343
Epoch 86
Validation prc-auc = 0.208891
Epoch 87
Validation prc-auc = 0.189567
Epoch 88
Validation prc-auc = 0.199458
Epoch 89
Loss = 3.3585e-02, PNorm = 84.2406, GNorm = 1.3216, lr_0 = 8.6941e-04
Validation prc-auc = 0.227136
Epoch 90
Validation prc-auc = 0.251822
Epoch 91
Validation prc-auc = 0.242586
Epoch 92
Validation prc-auc = 0.225761
Epoch 93
Validation prc-auc = 0.209611
Epoch 94
Loss = 1.7099e-02, PNorm = 84.4713, GNorm = 0.4374, lr_0 = 8.6198e-04
Validation prc-auc = 0.200627
Epoch 95
Validation prc-auc = 0.239579
Epoch 96
Validation prc-auc = 0.252516
Epoch 97
Validation prc-auc = 0.266980
Epoch 98
Validation prc-auc = 0.286916
Epoch 99
Loss = 7.2256e-03, PNorm = 84.6944, GNorm = 0.2185, lr_0 = 8.5461e-04
Validation prc-auc = 0.287583
Epoch 100
Validation prc-auc = 0.251571
Epoch 101
Validation prc-auc = 0.241074
Epoch 102
Validation prc-auc = 0.205227
Epoch 103
Validation prc-auc = 0.171208
Epoch 104
Loss = 5.0183e-02, PNorm = 84.8603, GNorm = 3.1848, lr_0 = 8.4730e-04
Validation prc-auc = 0.210539
Epoch 105
Validation prc-auc = 0.259053
Epoch 106
Validation prc-auc = 0.281646
Epoch 107
Validation prc-auc = 0.276828
Epoch 108
Validation prc-auc = 0.281906
Epoch 109
Loss = 1.3714e-02, PNorm = 85.0971, GNorm = 0.2898, lr_0 = 8.4006e-04
Validation prc-auc = 0.301848
Epoch 110
Validation prc-auc = 0.310100
Epoch 111
Validation prc-auc = 0.299987
Epoch 112
Validation prc-auc = 0.247440
Epoch 113
Validation prc-auc = 0.216580
Epoch 114
Loss = 1.5790e-02, PNorm = 85.3088, GNorm = 0.0343, lr_0 = 8.3288e-04
Validation prc-auc = 0.197170
Epoch 115
Validation prc-auc = 0.193215
Epoch 116
Validation prc-auc = 0.207562
Epoch 117
Validation prc-auc = 0.219012
Epoch 118
Validation prc-auc = 0.227806
Epoch 119
Loss = 7.5661e-03, PNorm = 85.5044, GNorm = 0.2694, lr_0 = 8.2576e-04
Validation prc-auc = 0.224288
Epoch 120
Validation prc-auc = 0.251187
Epoch 121
Validation prc-auc = 0.226028
Epoch 122
Validation prc-auc = 0.220938
Epoch 123
Validation prc-auc = 0.228040
Epoch 124
Loss = 5.3320e-02, PNorm = 85.6556, GNorm = 1.0852, lr_0 = 8.1870e-04
Validation prc-auc = 0.196481
Epoch 125
Validation prc-auc = 0.205610
Epoch 126
Validation prc-auc = 0.236184
Epoch 127
Validation prc-auc = 0.241849
Epoch 128
Validation prc-auc = 0.259201
Epoch 129
Loss = 3.4204e-02, PNorm = 85.8841, GNorm = 1.9203, lr_0 = 8.1170e-04
Validation prc-auc = 0.260198
Epoch 130
Validation prc-auc = 0.256769
Epoch 131
Validation prc-auc = 0.205087
Epoch 132
Validation prc-auc = 0.194703
Epoch 133
Validation prc-auc = 0.222715
Epoch 134
Loss = 1.1991e-02, PNorm = 86.0542, GNorm = 0.4515, lr_0 = 8.0476e-04
Validation prc-auc = 0.240268
Epoch 135
Validation prc-auc = 0.257440
Epoch 136
Validation prc-auc = 0.261329
Epoch 137
Validation prc-auc = 0.262965
Epoch 138
Validation prc-auc = 0.301042
Epoch 139
Loss = 3.7627e-03, PNorm = 86.2992, GNorm = 0.1044, lr_0 = 7.9788e-04
Validation prc-auc = 0.311650
Epoch 140
Validation prc-auc = 0.320001
Epoch 141
Validation prc-auc = 0.306695
Epoch 142
Validation prc-auc = 0.270824
Epoch 143
Validation prc-auc = 0.253500
Epoch 144
Loss = 7.5574e-03, PNorm = 86.4422, GNorm = 0.2978, lr_0 = 7.9106e-04
Validation prc-auc = 0.244863
Epoch 145
Validation prc-auc = 0.241338
Epoch 146
Validation prc-auc = 0.240851
Epoch 147
Validation prc-auc = 0.241389
Epoch 148
Validation prc-auc = 0.257856
Epoch 149
Loss = 7.9625e-03, PNorm = 86.6006, GNorm = 0.6261, lr_0 = 7.8430e-04
Validation prc-auc = 0.252878
Epoch 150
Validation prc-auc = 0.247248
Epoch 151
Validation prc-auc = 0.250543
Epoch 152
Validation prc-auc = 0.210007
Epoch 153
Validation prc-auc = 0.196377
Epoch 154
Loss = 4.1447e-03, PNorm = 86.7082, GNorm = 0.1345, lr_0 = 7.7759e-04
Validation prc-auc = 0.181752
Epoch 155
Validation prc-auc = 0.175607
Epoch 156
Validation prc-auc = 0.172722
Epoch 157
Validation prc-auc = 0.192026
Epoch 158
Validation prc-auc = 0.201716
Epoch 159
Loss = 1.8512e-03, PNorm = 86.7960, GNorm = 0.0608, lr_0 = 7.7095e-04
Validation prc-auc = 0.215760
Epoch 160
Validation prc-auc = 0.221667
Epoch 161
Validation prc-auc = 0.220881
Epoch 162
Validation prc-auc = 0.226931
Epoch 163
Validation prc-auc = 0.226766
Epoch 164
Loss = 1.4032e-03, PNorm = 86.8615, GNorm = 0.0165, lr_0 = 7.6436e-04
Validation prc-auc = 0.230959
Epoch 165
Validation prc-auc = 0.230962
Epoch 166
Validation prc-auc = 0.231031
Epoch 167
Validation prc-auc = 0.230962
Epoch 168
Validation prc-auc = 0.231195
Epoch 169
Loss = 8.6651e-04, PNorm = 86.8980, GNorm = 0.0151, lr_0 = 7.5782e-04
Validation prc-auc = 0.232348
Epoch 170
Validation prc-auc = 0.233524
Epoch 171
Validation prc-auc = 0.233524
Epoch 172
Validation prc-auc = 0.231752
Epoch 173
Validation prc-auc = 0.231915
Epoch 174
Loss = 3.3568e-04, PNorm = 86.9257, GNorm = 0.0100, lr_0 = 7.5134e-04
Validation prc-auc = 0.230779
Epoch 175
Validation prc-auc = 0.230578
Epoch 176
Validation prc-auc = 0.229896
Epoch 177
Validation prc-auc = 0.227245
Epoch 178
Validation prc-auc = 0.231205
Epoch 179
Loss = 1.4119e-03, PNorm = 86.9472, GNorm = 0.1662, lr_0 = 7.4492e-04
Validation prc-auc = 0.233057
Epoch 180
Validation prc-auc = 0.229963
Epoch 181
Validation prc-auc = 0.230312
Epoch 182
Validation prc-auc = 0.232165
Epoch 183
Validation prc-auc = 0.232298
Epoch 184
Loss = 4.3759e-04, PNorm = 86.9597, GNorm = 0.0093, lr_0 = 7.3855e-04
Validation prc-auc = 0.231458
Epoch 185
Validation prc-auc = 0.231291
Epoch 186
Validation prc-auc = 0.231148
Epoch 187
Validation prc-auc = 0.231077
Epoch 188
Validation prc-auc = 0.230981
Epoch 189
Loss = 3.5932e-04, PNorm = 86.9807, GNorm = 0.0078, lr_0 = 7.3224e-04
Validation prc-auc = 0.230504
Epoch 190
Validation prc-auc = 0.226111
Epoch 191
Validation prc-auc = 0.226568
Epoch 192
Validation prc-auc = 0.225758
Epoch 193
Validation prc-auc = 0.225825
Epoch 194
Loss = 3.8743e-04, PNorm = 86.9999, GNorm = 0.0089, lr_0 = 7.2598e-04
Validation prc-auc = 0.225745
Epoch 195
Validation prc-auc = 0.228954
Epoch 196
Validation prc-auc = 0.228890
Epoch 197
Validation prc-auc = 0.230440
Epoch 198
Validation prc-auc = 0.230437
Epoch 199
Loss = 1.7005e-04, PNorm = 87.0120, GNorm = 0.0049, lr_0 = 7.1977e-04
Validation prc-auc = 0.231265
Epoch 200
Validation prc-auc = 0.231446
Epoch 201
Validation prc-auc = 0.231379
Epoch 202
Validation prc-auc = 0.235159
Epoch 203
Validation prc-auc = 0.235159
Epoch 204
Loss = 1.4709e-04, PNorm = 87.0239, GNorm = 0.0044, lr_0 = 7.1362e-04
Validation prc-auc = 0.235159
Epoch 205
Validation prc-auc = 0.238190
Epoch 206
Validation prc-auc = 0.238191
Epoch 207
Validation prc-auc = 0.235292
Epoch 208
Validation prc-auc = 0.235292
Epoch 209
Loss = 2.4106e-04, PNorm = 87.0373, GNorm = 0.0197, lr_0 = 7.0752e-04
Validation prc-auc = 0.235292
Epoch 210
Validation prc-auc = 0.235228
Epoch 211
Validation prc-auc = 0.235247
Epoch 212
Validation prc-auc = 0.243912
Epoch 213
Validation prc-auc = 0.245091
Epoch 214
Loss = 8.8845e-05, PNorm = 87.0463, GNorm = 0.0031, lr_0 = 7.0147e-04
Validation prc-auc = 0.245023
Epoch 215
Validation prc-auc = 0.245093
Epoch 216
Validation prc-auc = 0.245093
Epoch 217
Validation prc-auc = 0.247590
Epoch 218
Validation prc-auc = 0.247634
Epoch 219
Loss = 1.6314e-04, PNorm = 87.0580, GNorm = 0.0022, lr_0 = 6.9548e-04
Validation prc-auc = 0.246717
Epoch 220
Validation prc-auc = 0.246717
Epoch 221
Validation prc-auc = 0.246717
Epoch 222
Validation prc-auc = 0.251977
Epoch 223
Validation prc-auc = 0.251886
Epoch 224
Loss = 5.7891e-04, PNorm = 87.0689, GNorm = 0.0141, lr_0 = 6.8953e-04
Validation prc-auc = 0.252070
Epoch 225
Validation prc-auc = 0.252165
Epoch 226
Validation prc-auc = 0.252196
Epoch 227
Validation prc-auc = 0.252196
Epoch 228
Validation prc-auc = 0.252120
Epoch 229
Loss = 9.1246e-05, PNorm = 87.0767, GNorm = 0.0028, lr_0 = 6.8364e-04
Validation prc-auc = 0.252186
Epoch 230
Validation prc-auc = 0.252196
Epoch 231
Validation prc-auc = 0.252196
Epoch 232
Validation prc-auc = 0.252196
Epoch 233
Validation prc-auc = 0.259637
Epoch 234
Loss = 1.0015e-04, PNorm = 87.0887, GNorm = 0.0019, lr_0 = 6.7779e-04
Validation prc-auc = 0.259478
Epoch 235
Validation prc-auc = 0.259461
Epoch 236
Validation prc-auc = 0.259640
Epoch 237
Validation prc-auc = 0.259782
Epoch 238
Validation prc-auc = 0.260681
Epoch 239
Loss = 6.6957e-05, PNorm = 87.0977, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.260835
Epoch 240
Validation prc-auc = 0.260910
Epoch 241
Validation prc-auc = 0.264819
Epoch 242
Validation prc-auc = 0.265838
Epoch 243
Validation prc-auc = 0.265932
Epoch 244
Loss = 5.9781e-05, PNorm = 87.1072, GNorm = 0.0013, lr_0 = 6.6625e-04
Validation prc-auc = 0.265996
Epoch 245
Validation prc-auc = 0.265902
Epoch 246
Validation prc-auc = 0.264921
Epoch 247
Validation prc-auc = 0.264822
Epoch 248
Validation prc-auc = 0.264643
Epoch 249
Loss = 4.1430e-05, PNorm = 87.1184, GNorm = 0.0010, lr_0 = 6.6056e-04
Validation prc-auc = 0.263816
Epoch 250
Validation prc-auc = 0.264666
Epoch 251
Validation prc-auc = 0.263755
Epoch 252
Validation prc-auc = 0.263099
Epoch 253
Validation prc-auc = 0.263928
Epoch 254
Loss = 4.1046e-05, PNorm = 87.1295, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.264008
Epoch 255
Validation prc-auc = 0.265221
Epoch 256
Validation prc-auc = 0.266494
Epoch 257
Validation prc-auc = 0.266876
Epoch 258
Validation prc-auc = 0.266876
Epoch 259
Loss = 4.6054e-05, PNorm = 87.1382, GNorm = 0.0006, lr_0 = 6.4931e-04
Validation prc-auc = 0.265720
Epoch 260
Validation prc-auc = 0.264760
Epoch 261
Validation prc-auc = 0.265828
Epoch 262
Validation prc-auc = 0.267399
Epoch 263
Validation prc-auc = 0.268491
Epoch 264
Loss = 2.0171e-05, PNorm = 87.1470, GNorm = 0.0006, lr_0 = 6.4376e-04
Validation prc-auc = 0.268647
Epoch 265
Validation prc-auc = 0.268741
Epoch 266
Validation prc-auc = 0.267219
Epoch 267
Validation prc-auc = 0.268426
Epoch 268
Validation prc-auc = 0.263798
Epoch 269
Loss = 1.1883e-05, PNorm = 87.1543, GNorm = 0.0009, lr_0 = 6.3826e-04
Validation prc-auc = 0.262209
Epoch 270
Validation prc-auc = 0.263728
Epoch 271
Validation prc-auc = 0.267963
Epoch 272
Validation prc-auc = 0.276863
Epoch 273
Validation prc-auc = 0.281979
Epoch 274
Loss = 1.8071e-05, PNorm = 87.1597, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.282208
Epoch 275
Validation prc-auc = 0.273036
Epoch 276
Validation prc-auc = 0.271376
Epoch 277
Validation prc-auc = 0.270654
Epoch 278
Validation prc-auc = 0.267815
Epoch 279
Loss = 1.2040e-05, PNorm = 87.1653, GNorm = 0.0022, lr_0 = 6.2739e-04
Validation prc-auc = 0.254367
Epoch 280
Validation prc-auc = 0.248043
Epoch 281
Validation prc-auc = 0.254376
Epoch 282
Validation prc-auc = 0.255854
Epoch 283
Validation prc-auc = 0.257660
Epoch 284
Loss = 1.3937e-05, PNorm = 87.1688, GNorm = 0.0005, lr_0 = 6.2203e-04
Validation prc-auc = 0.261545
Epoch 285
Validation prc-auc = 0.264887
Epoch 286
Validation prc-auc = 0.264821
Epoch 287
Validation prc-auc = 0.264757
Epoch 288
Validation prc-auc = 0.262994
Epoch 289
Loss = 7.4108e-06, PNorm = 87.1729, GNorm = 0.0005, lr_0 = 6.1671e-04
Validation prc-auc = 0.263275
Epoch 290
Validation prc-auc = 0.262037
Epoch 291
Validation prc-auc = 0.262037
Epoch 292
Validation prc-auc = 0.261647
Epoch 293
Validation prc-auc = 0.261562
Epoch 294
Loss = 8.6336e-06, PNorm = 87.1768, GNorm = 0.0006, lr_0 = 6.1144e-04
Validation prc-auc = 0.257813
Epoch 295
Validation prc-auc = 0.280159
Epoch 296
Validation prc-auc = 0.282587
Epoch 297
Validation prc-auc = 0.285804
Epoch 298
Validation prc-auc = 0.282537
Epoch 299
Loss = 2.0865e-05, PNorm = 87.1799, GNorm = 0.0019, lr_0 = 6.0621e-04
Validation prc-auc = 0.281201
Model 0 best validation prc-auc = 0.472303 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387643
Ensemble test prc-auc = 0.387643
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.434076
Epoch 1
Validation prc-auc = 0.457420
Epoch 2
Validation prc-auc = 0.458604
Epoch 3
Validation prc-auc = 0.471728
Epoch 4
Loss = 6.6940e-01, PNorm = 78.5476, GNorm = 6.7897, lr_0 = 6.5000e-04
Validation prc-auc = 0.459812
Epoch 5
Validation prc-auc = 0.441032
Epoch 6
Validation prc-auc = 0.481371
Epoch 7
Validation prc-auc = 0.465603
Epoch 8
Validation prc-auc = 0.490614
Epoch 9
Loss = 7.5357e-01, PNorm = 78.7818, GNorm = 4.4542, lr_0 = 9.9743e-04
Validation prc-auc = 0.478565
Epoch 10
Validation prc-auc = 0.321278
Epoch 11
Validation prc-auc = 0.329693
Epoch 12
Validation prc-auc = 0.402455
Epoch 13
Validation prc-auc = 0.457976
Epoch 14
Loss = 4.9512e-01, PNorm = 79.0603, GNorm = 4.6236, lr_0 = 9.8890e-04
Validation prc-auc = 0.465991
Epoch 15
Validation prc-auc = 0.474146
Epoch 16
Validation prc-auc = 0.494079
Epoch 17
Validation prc-auc = 0.481656
Epoch 18
Validation prc-auc = 0.491480
Epoch 19
Loss = 6.0016e-01, PNorm = 79.3179, GNorm = 11.3375, lr_0 = 9.8045e-04
Validation prc-auc = 0.356324
Epoch 20
Validation prc-auc = 0.319028
Epoch 21
Validation prc-auc = 0.396666
Epoch 22
Validation prc-auc = 0.327262
Epoch 23
Validation prc-auc = 0.249020
Epoch 24
Loss = 5.2568e-01, PNorm = 79.5741, GNorm = 3.9257, lr_0 = 9.7207e-04
Validation prc-auc = 0.344926
Epoch 25
Validation prc-auc = 0.463539
Epoch 26
Validation prc-auc = 0.462075
Epoch 27
Validation prc-auc = 0.463508
Epoch 28
Validation prc-auc = 0.462585
Epoch 29
Loss = 4.8583e-01, PNorm = 79.8787, GNorm = 1.1889, lr_0 = 9.6376e-04
Validation prc-auc = 0.441641
Epoch 30
Validation prc-auc = 0.350212
Epoch 31
Validation prc-auc = 0.307020
Epoch 32
Validation prc-auc = 0.295028
Epoch 33
Validation prc-auc = 0.308890
Epoch 34
Loss = 3.5980e-01, PNorm = 80.1620, GNorm = 1.2794, lr_0 = 9.5552e-04
Validation prc-auc = 0.347034
Epoch 35
Validation prc-auc = 0.345844
Epoch 36
Validation prc-auc = 0.329048
Epoch 37
Validation prc-auc = 0.338035
Epoch 38
Validation prc-auc = 0.385873
Epoch 39
Loss = 3.6370e-01, PNorm = 80.4752, GNorm = 3.3556, lr_0 = 9.4735e-04
Validation prc-auc = 0.354778
Epoch 40
Validation prc-auc = 0.310202
Epoch 41
Validation prc-auc = 0.297669
Epoch 42
Validation prc-auc = 0.324231
Epoch 43
Validation prc-auc = 0.296734
Epoch 44
Loss = 3.6850e-01, PNorm = 80.7925, GNorm = 5.2399, lr_0 = 9.3925e-04
Validation prc-auc = 0.279638
Epoch 45
Validation prc-auc = 0.334626
Epoch 46
Validation prc-auc = 0.341671
Epoch 47
Validation prc-auc = 0.358256
Epoch 48
Validation prc-auc = 0.330206
Epoch 49
Loss = 2.1353e-01, PNorm = 81.1628, GNorm = 2.3248, lr_0 = 9.3122e-04
Validation prc-auc = 0.352046
Epoch 50
Validation prc-auc = 0.303014
Epoch 51
Validation prc-auc = 0.310926
Epoch 52
Validation prc-auc = 0.307721
Epoch 53
Validation prc-auc = 0.320798
Epoch 54
Loss = 2.5593e-01, PNorm = 81.5512, GNorm = 5.2760, lr_0 = 9.2326e-04
Validation prc-auc = 0.343211
Epoch 55
Validation prc-auc = 0.337257
Epoch 56
Validation prc-auc = 0.359772
Epoch 57
Validation prc-auc = 0.377172
Epoch 58
Validation prc-auc = 0.342004
Epoch 59
Loss = 1.8163e-01, PNorm = 81.9307, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation prc-auc = 0.298545
Epoch 60
Validation prc-auc = 0.280353
Epoch 61
Validation prc-auc = 0.294281
Epoch 62
Validation prc-auc = 0.265543
Epoch 63
Validation prc-auc = 0.258436
Epoch 64
Loss = 1.3860e-01, PNorm = 82.3406, GNorm = 1.8254, lr_0 = 9.0754e-04
Validation prc-auc = 0.274408
Epoch 65
Validation prc-auc = 0.191392
Epoch 66
Validation prc-auc = 0.298554
Epoch 67
Validation prc-auc = 0.304117
Epoch 68
Validation prc-auc = 0.362464
Epoch 69
Loss = 6.8738e-02, PNorm = 82.7452, GNorm = 1.6550, lr_0 = 8.9978e-04
Validation prc-auc = 0.373048
Epoch 70
Validation prc-auc = 0.370687
Epoch 71
Validation prc-auc = 0.368884
Epoch 72
Validation prc-auc = 0.334850
Epoch 73
Validation prc-auc = 0.307704
Epoch 74
Loss = 6.8877e-02, PNorm = 83.0944, GNorm = 0.3143, lr_0 = 8.9209e-04
Validation prc-auc = 0.269460
Epoch 75
Validation prc-auc = 0.229206
Epoch 76
Validation prc-auc = 0.269542
Epoch 77
Validation prc-auc = 0.207595
Epoch 78
Validation prc-auc = 0.290756
Epoch 79
Loss = 8.6926e-02, PNorm = 83.4015, GNorm = 0.5161, lr_0 = 8.8447e-04
Validation prc-auc = 0.283063
Epoch 80
Validation prc-auc = 0.283195
Epoch 81
Validation prc-auc = 0.328266
Epoch 82
Validation prc-auc = 0.342782
Epoch 83
Validation prc-auc = 0.306091
Epoch 84
Loss = 2.3302e-01, PNorm = 83.7335, GNorm = 1.7786, lr_0 = 8.7691e-04
Validation prc-auc = 0.279411
Epoch 85
Validation prc-auc = 0.281117
Epoch 86
Validation prc-auc = 0.246712
Epoch 87
Validation prc-auc = 0.224165
Epoch 88
Validation prc-auc = 0.287092
Epoch 89
Loss = 4.5509e-02, PNorm = 84.1042, GNorm = 0.6834, lr_0 = 8.6941e-04
Validation prc-auc = 0.308344
Epoch 90
Validation prc-auc = 0.328625
Epoch 91
Validation prc-auc = 0.309906
Epoch 92
Validation prc-auc = 0.261949
Epoch 93
Validation prc-auc = 0.238334
Epoch 94
Loss = 2.9086e-02, PNorm = 84.3840, GNorm = 0.7699, lr_0 = 8.6198e-04
Validation prc-auc = 0.233986
Epoch 95
Validation prc-auc = 0.242076
Epoch 96
Validation prc-auc = 0.234171
Epoch 97
Validation prc-auc = 0.259769
Epoch 98
Validation prc-auc = 0.260372
Epoch 99
Loss = 2.7894e-02, PNorm = 84.6258, GNorm = 0.5002, lr_0 = 8.5461e-04
Validation prc-auc = 0.241337
Epoch 100
Validation prc-auc = 0.230159
Epoch 101
Validation prc-auc = 0.240429
Epoch 102
Validation prc-auc = 0.245848
Epoch 103
Validation prc-auc = 0.256310
Epoch 104
Loss = 4.0704e-02, PNorm = 84.8366, GNorm = 1.1440, lr_0 = 8.4730e-04
Validation prc-auc = 0.229493
Epoch 105
Validation prc-auc = 0.226121
Epoch 106
Validation prc-auc = 0.222959
Epoch 107
Validation prc-auc = 0.226675
Epoch 108
Validation prc-auc = 0.242013
Epoch 109
Loss = 1.4584e-02, PNorm = 84.9962, GNorm = 0.6123, lr_0 = 8.4006e-04
Validation prc-auc = 0.256096
Epoch 110
Validation prc-auc = 0.262724
Epoch 111
Validation prc-auc = 0.237542
Epoch 112
Validation prc-auc = 0.212855
Epoch 113
Validation prc-auc = 0.222727
Epoch 114
Loss = 2.4388e-02, PNorm = 85.1564, GNorm = 0.9167, lr_0 = 8.3288e-04
Validation prc-auc = 0.242274
Epoch 115
Validation prc-auc = 0.279204
Epoch 116
Validation prc-auc = 0.278679
Epoch 117
Validation prc-auc = 0.253609
Epoch 118
Validation prc-auc = 0.214742
Epoch 119
Loss = 3.5193e-02, PNorm = 85.3554, GNorm = 0.8935, lr_0 = 8.2576e-04
Validation prc-auc = 0.189706
Epoch 120
Validation prc-auc = 0.186311
Epoch 121
Validation prc-auc = 0.245165
Epoch 122
Validation prc-auc = 0.237705
Epoch 123
Validation prc-auc = 0.240659
Epoch 124
Loss = 2.4166e-02, PNorm = 85.5488, GNorm = 0.2369, lr_0 = 8.1870e-04
Validation prc-auc = 0.250475
Epoch 125
Validation prc-auc = 0.266610
Epoch 126
Validation prc-auc = 0.266784
Epoch 127
Validation prc-auc = 0.270365
Epoch 128
Validation prc-auc = 0.245779
Epoch 129
Loss = 7.9705e-03, PNorm = 85.7155, GNorm = 0.1555, lr_0 = 8.1170e-04
Validation prc-auc = 0.231842
Epoch 130
Validation prc-auc = 0.214591
Epoch 131
Validation prc-auc = 0.212922
Epoch 132
Validation prc-auc = 0.210431
Epoch 133
Validation prc-auc = 0.209643
Epoch 134
Loss = 3.5030e-03, PNorm = 85.8338, GNorm = 0.2374, lr_0 = 8.0476e-04
Validation prc-auc = 0.207306
Epoch 135
Validation prc-auc = 0.200479
Epoch 136
Validation prc-auc = 0.202470
Epoch 137
Validation prc-auc = 0.205772
Epoch 138
Validation prc-auc = 0.209165
Epoch 139
Loss = 3.8180e-03, PNorm = 85.9040, GNorm = 0.2716, lr_0 = 7.9788e-04
Validation prc-auc = 0.213847
Epoch 140
Validation prc-auc = 0.218795
Epoch 141
Validation prc-auc = 0.210443
Epoch 142
Validation prc-auc = 0.219385
Epoch 143
Validation prc-auc = 0.244349
Epoch 144
Loss = 1.2857e-02, PNorm = 85.9696, GNorm = 0.3995, lr_0 = 7.9106e-04
Validation prc-auc = 0.237443
Epoch 145
Validation prc-auc = 0.246441
Epoch 146
Validation prc-auc = 0.253707
Epoch 147
Validation prc-auc = 0.269453
Epoch 148
Validation prc-auc = 0.283431
Epoch 149
Loss = 8.9761e-03, PNorm = 86.1367, GNorm = 0.2976, lr_0 = 7.8430e-04
Validation prc-auc = 0.289997
Epoch 150
Validation prc-auc = 0.295357
Epoch 151
Validation prc-auc = 0.261798
Epoch 152
Validation prc-auc = 0.235727
Epoch 153
Validation prc-auc = 0.247289
Epoch 154
Loss = 1.5452e-02, PNorm = 86.2802, GNorm = 0.1277, lr_0 = 7.7759e-04
Validation prc-auc = 0.241072
Epoch 155
Validation prc-auc = 0.242246
Epoch 156
Validation prc-auc = 0.214299
Epoch 157
Validation prc-auc = 0.199380
Epoch 158
Validation prc-auc = 0.239951
Epoch 159
Loss = 5.9379e-03, PNorm = 86.4244, GNorm = 0.1018, lr_0 = 7.7095e-04
Validation prc-auc = 0.246812
Epoch 160
Validation prc-auc = 0.255179
Epoch 161
Validation prc-auc = 0.248417
Epoch 162
Validation prc-auc = 0.238683
Epoch 163
Validation prc-auc = 0.237195
Epoch 164
Loss = 1.9421e-03, PNorm = 86.5171, GNorm = 0.0091, lr_0 = 7.6436e-04
Validation prc-auc = 0.250266
Epoch 165
Validation prc-auc = 0.252963
Epoch 166
Validation prc-auc = 0.254370
Epoch 167
Validation prc-auc = 0.250415
Epoch 168
Validation prc-auc = 0.249695
Epoch 169
Loss = 1.3199e-03, PNorm = 86.5762, GNorm = 0.0447, lr_0 = 7.5782e-04
Validation prc-auc = 0.249349
Epoch 170
Validation prc-auc = 0.246538
Epoch 171
Validation prc-auc = 0.240236
Epoch 172
Validation prc-auc = 0.240534
Epoch 173
Validation prc-auc = 0.235577
Epoch 174
Loss = 2.2461e-03, PNorm = 86.6135, GNorm = 0.2446, lr_0 = 7.5134e-04
Validation prc-auc = 0.244731
Epoch 175
Validation prc-auc = 0.246985
Epoch 176
Validation prc-auc = 0.249017
Epoch 177
Validation prc-auc = 0.252572
Epoch 178
Validation prc-auc = 0.251968
Epoch 179
Loss = 6.3653e-04, PNorm = 86.6483, GNorm = 0.0177, lr_0 = 7.4492e-04
Validation prc-auc = 0.249646
Epoch 180
Validation prc-auc = 0.242470
Epoch 181
Validation prc-auc = 0.243353
Epoch 182
Validation prc-auc = 0.243127
Epoch 183
Validation prc-auc = 0.240641
Epoch 184
Loss = 4.6618e-04, PNorm = 86.6758, GNorm = 0.0062, lr_0 = 7.3855e-04
Validation prc-auc = 0.237657
Epoch 185
Validation prc-auc = 0.237527
Epoch 186
Validation prc-auc = 0.239330
Epoch 187
Validation prc-auc = 0.238970
Epoch 188
Validation prc-auc = 0.243761
Epoch 189
Loss = 1.4330e-03, PNorm = 86.6921, GNorm = 0.0315, lr_0 = 7.3224e-04
Validation prc-auc = 0.249133
Epoch 190
Validation prc-auc = 0.253101
Epoch 191
Validation prc-auc = 0.245300
Epoch 192
Validation prc-auc = 0.252355
Epoch 193
Validation prc-auc = 0.253956
Epoch 194
Loss = 3.3508e-04, PNorm = 86.7066, GNorm = 0.0125, lr_0 = 7.2598e-04
Validation prc-auc = 0.253320
Epoch 195
Validation prc-auc = 0.253719
Epoch 196
Validation prc-auc = 0.254532
Epoch 197
Validation prc-auc = 0.254145
Epoch 198
Validation prc-auc = 0.254142
Epoch 199
Loss = 3.3925e-04, PNorm = 86.7180, GNorm = 0.0175, lr_0 = 7.1977e-04
Validation prc-auc = 0.254220
Epoch 200
Validation prc-auc = 0.254019
Epoch 201
Validation prc-auc = 0.254220
Epoch 202
Validation prc-auc = 0.254220
Epoch 203
Validation prc-auc = 0.253649
Epoch 204
Loss = 1.5806e-04, PNorm = 86.7262, GNorm = 0.0059, lr_0 = 7.1362e-04
Validation prc-auc = 0.252956
Epoch 205
Validation prc-auc = 0.252673
Epoch 206
Validation prc-auc = 0.252604
Epoch 207
Validation prc-auc = 0.252604
Epoch 208
Validation prc-auc = 0.252525
Epoch 209
Loss = 8.6087e-05, PNorm = 86.7315, GNorm = 0.0030, lr_0 = 7.0752e-04
Validation prc-auc = 0.250358
Epoch 210
Validation prc-auc = 0.253926
Epoch 211
Validation prc-auc = 0.253731
Epoch 212
Validation prc-auc = 0.254005
Epoch 213
Validation prc-auc = 0.254205
Epoch 214
Loss = 2.0440e-04, PNorm = 86.7374, GNorm = 0.0069, lr_0 = 7.0147e-04
Validation prc-auc = 0.254903
Epoch 215
Validation prc-auc = 0.257509
Epoch 216
Validation prc-auc = 0.257265
Epoch 217
Validation prc-auc = 0.257896
Epoch 218
Validation prc-auc = 0.257896
Epoch 219
Loss = 1.3274e-04, PNorm = 86.7445, GNorm = 0.0034, lr_0 = 6.9548e-04
Validation prc-auc = 0.257900
Epoch 220
Validation prc-auc = 0.252493
Epoch 221
Validation prc-auc = 0.252493
Epoch 222
Validation prc-auc = 0.252430
Epoch 223
Validation prc-auc = 0.252665
Epoch 224
Loss = 2.3222e-04, PNorm = 86.7508, GNorm = 0.0105, lr_0 = 6.8953e-04
Validation prc-auc = 0.251845
Epoch 225
Validation prc-auc = 0.251845
Epoch 226
Validation prc-auc = 0.251988
Epoch 227
Validation prc-auc = 0.256996
Epoch 228
Validation prc-auc = 0.256713
Epoch 229
Loss = 1.0546e-04, PNorm = 86.7573, GNorm = 0.0036, lr_0 = 6.8364e-04
Validation prc-auc = 0.256713
Epoch 230
Validation prc-auc = 0.256467
Epoch 231
Validation prc-auc = 0.256467
Epoch 232
Validation prc-auc = 0.256197
Epoch 233
Validation prc-auc = 0.255886
Epoch 234
Loss = 1.0012e-04, PNorm = 86.7625, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.253234
Epoch 235
Validation prc-auc = 0.250456
Epoch 236
Validation prc-auc = 0.249570
Epoch 237
Validation prc-auc = 0.249646
Epoch 238
Validation prc-auc = 0.249646
Epoch 239
Loss = 1.4534e-04, PNorm = 86.7672, GNorm = 0.0069, lr_0 = 6.7200e-04
Validation prc-auc = 0.249527
Epoch 240
Validation prc-auc = 0.248775
Epoch 241
Validation prc-auc = 0.250517
Epoch 242
Validation prc-auc = 0.250440
Epoch 243
Validation prc-auc = 0.250516
Epoch 244
Loss = 1.7778e-04, PNorm = 86.7717, GNorm = 0.0076, lr_0 = 6.6625e-04
Validation prc-auc = 0.250440
Epoch 245
Validation prc-auc = 0.249983
Epoch 246
Validation prc-auc = 0.249986
Epoch 247
Validation prc-auc = 0.249986
Epoch 248
Validation prc-auc = 0.250038
Epoch 249
Loss = 5.5124e-05, PNorm = 86.7762, GNorm = 0.0019, lr_0 = 6.6056e-04
Validation prc-auc = 0.250105
Epoch 250
Validation prc-auc = 0.249858
Epoch 251
Validation prc-auc = 0.249911
Epoch 252
Validation prc-auc = 0.249723
Epoch 253
Validation prc-auc = 0.250603
Epoch 254
Loss = 6.4436e-05, PNorm = 86.7802, GNorm = 0.0013, lr_0 = 6.5491e-04
Validation prc-auc = 0.249911
Epoch 255
Validation prc-auc = 0.249990
Epoch 256
Validation prc-auc = 0.249803
Epoch 257
Validation prc-auc = 0.249567
Epoch 258
Validation prc-auc = 0.249567
Epoch 259
Loss = 9.6219e-05, PNorm = 86.7837, GNorm = 0.0085, lr_0 = 6.4931e-04
Validation prc-auc = 0.249567
Epoch 260
Validation prc-auc = 0.249635
Epoch 261
Validation prc-auc = 0.249822
Epoch 262
Validation prc-auc = 0.250218
Epoch 263
Validation prc-auc = 0.250218
Epoch 264
Loss = 5.8861e-05, PNorm = 86.7876, GNorm = 0.0022, lr_0 = 6.4376e-04
Validation prc-auc = 0.250150
Epoch 265
Validation prc-auc = 0.250150
Epoch 266
Validation prc-auc = 0.249923
Epoch 267
Validation prc-auc = 0.256431
Epoch 268
Validation prc-auc = 0.256215
Epoch 269
Loss = 1.0507e-04, PNorm = 86.7916, GNorm = 0.0065, lr_0 = 6.3826e-04
Validation prc-auc = 0.256013
Epoch 270
Validation prc-auc = 0.256090
Epoch 271
Validation prc-auc = 0.256090
Epoch 272
Validation prc-auc = 0.256090
Epoch 273
Validation prc-auc = 0.256090
Epoch 274
Loss = 5.5034e-05, PNorm = 86.7959, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.256090
Epoch 275
Validation prc-auc = 0.255896
Epoch 276
Validation prc-auc = 0.255819
Epoch 277
Validation prc-auc = 0.255611
Epoch 278
Validation prc-auc = 0.255611
Epoch 279
Loss = 7.1451e-05, PNorm = 86.8009, GNorm = 0.0042, lr_0 = 6.2739e-04
Validation prc-auc = 0.255611
Epoch 280
Validation prc-auc = 0.255597
Epoch 281
Validation prc-auc = 0.255597
Epoch 282
Validation prc-auc = 0.255397
Epoch 283
Validation prc-auc = 0.254757
Epoch 284
Loss = 1.0247e-04, PNorm = 86.8062, GNorm = 0.0051, lr_0 = 6.2203e-04
Validation prc-auc = 0.254957
Epoch 285
Validation prc-auc = 0.255151
Epoch 286
Validation prc-auc = 0.255353
Epoch 287
Validation prc-auc = 0.255629
Epoch 288
Validation prc-auc = 0.255629
Epoch 289
Loss = 5.7148e-05, PNorm = 86.8121, GNorm = 0.0029, lr_0 = 6.1671e-04
Validation prc-auc = 0.255706
Epoch 290
Validation prc-auc = 0.256346
Epoch 291
Validation prc-auc = 0.256346
Epoch 292
Validation prc-auc = 0.256063
Epoch 293
Validation prc-auc = 0.255600
Epoch 294
Loss = 5.4535e-05, PNorm = 86.8190, GNorm = 0.0005, lr_0 = 6.1144e-04
Validation prc-auc = 0.255335
Epoch 295
Validation prc-auc = 0.255335
Epoch 296
Validation prc-auc = 0.255335
Epoch 297
Validation prc-auc = 0.255143
Epoch 298
Validation prc-auc = 0.255143
Epoch 299
Loss = 4.1452e-05, PNorm = 86.8257, GNorm = 0.0015, lr_0 = 6.0621e-04
Validation prc-auc = 0.255216
Model 0 best validation prc-auc = 0.494079 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.473718
Ensemble test prc-auc = 0.473718
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.444433
Epoch 1
Validation prc-auc = 0.456129
Epoch 2
Validation prc-auc = 0.420647
Epoch 3
Validation prc-auc = 0.440192
Epoch 4
Loss = 6.5323e-01, PNorm = 78.5536, GNorm = 3.8553, lr_0 = 6.5000e-04
Validation prc-auc = 0.469897
Epoch 5
Validation prc-auc = 0.441008
Epoch 6
Validation prc-auc = 0.439744
Epoch 7
Validation prc-auc = 0.429229
Epoch 8
Validation prc-auc = 0.430780
Epoch 9
Loss = 6.5283e-01, PNorm = 78.7980, GNorm = 8.9798, lr_0 = 9.9743e-04
Validation prc-auc = 0.299335
Epoch 10
Validation prc-auc = 0.310533
Epoch 11
Validation prc-auc = 0.376633
Epoch 12
Validation prc-auc = 0.478044
Epoch 13
Validation prc-auc = 0.421652
Epoch 14
Loss = 5.3995e-01, PNorm = 79.1048, GNorm = 0.7729, lr_0 = 9.8890e-04
Validation prc-auc = 0.412349
Epoch 15
Validation prc-auc = 0.403937
Epoch 16
Validation prc-auc = 0.487608
Epoch 17
Validation prc-auc = 0.466166
Epoch 18
Validation prc-auc = 0.297784
Epoch 19
Loss = 4.1483e-01, PNorm = 79.4444, GNorm = 3.4478, lr_0 = 9.8045e-04
Validation prc-auc = 0.271431
Epoch 20
Validation prc-auc = 0.322065
Epoch 21
Validation prc-auc = 0.319643
Epoch 22
Validation prc-auc = 0.320843
Epoch 23
Validation prc-auc = 0.268977
Epoch 24
Loss = 5.0285e-01, PNorm = 79.8039, GNorm = 4.1674, lr_0 = 9.7207e-04
Validation prc-auc = 0.289349
Epoch 25
Validation prc-auc = 0.392423
Epoch 26
Validation prc-auc = 0.275621
Epoch 27
Validation prc-auc = 0.306595
Epoch 28
Validation prc-auc = 0.353579
Epoch 29
Loss = 4.1142e-01, PNorm = 80.2021, GNorm = 2.4667, lr_0 = 9.6376e-04
Validation prc-auc = 0.349656
Epoch 30
Validation prc-auc = 0.418446
Epoch 31
Validation prc-auc = 0.441640
Epoch 32
Validation prc-auc = 0.361011
Epoch 33
Validation prc-auc = 0.355823
Epoch 34
Loss = 3.6353e-01, PNorm = 80.5821, GNorm = 7.7298, lr_0 = 9.5552e-04
Validation prc-auc = 0.330606
Epoch 35
Validation prc-auc = 0.365648
Epoch 36
Validation prc-auc = 0.336102
Epoch 37
Validation prc-auc = 0.361568
Epoch 38
Validation prc-auc = 0.382436
Epoch 39
Loss = 2.5884e-01, PNorm = 81.0056, GNorm = 2.0205, lr_0 = 9.4735e-04
Validation prc-auc = 0.399115
Epoch 40
Validation prc-auc = 0.387232
Epoch 41
Validation prc-auc = 0.351931
Epoch 42
Validation prc-auc = 0.235007
Epoch 43
Validation prc-auc = 0.243459
Epoch 44
Loss = 1.6512e-01, PNorm = 81.4481, GNorm = 1.2873, lr_0 = 9.3925e-04
Validation prc-auc = 0.226750
Epoch 45
Validation prc-auc = 0.359088
Epoch 46
Validation prc-auc = 0.358515
Epoch 47
Validation prc-auc = 0.339205
Epoch 48
Validation prc-auc = 0.335852
Epoch 49
Loss = 1.2805e-01, PNorm = 81.9343, GNorm = 1.6719, lr_0 = 9.3122e-04
Validation prc-auc = 0.320263
Epoch 50
Validation prc-auc = 0.314936
Epoch 51
Validation prc-auc = 0.317689
Epoch 52
Validation prc-auc = 0.315497
Epoch 53
Validation prc-auc = 0.296828
Epoch 54
Loss = 8.5127e-02, PNorm = 82.3601, GNorm = 0.4456, lr_0 = 9.2326e-04
Validation prc-auc = 0.182812
Epoch 55
Validation prc-auc = 0.227757
Epoch 56
Validation prc-auc = 0.256377
Epoch 57
Validation prc-auc = 0.194120
Epoch 58
Validation prc-auc = 0.242637
Epoch 59
Loss = 7.3885e-02, PNorm = 82.7524, GNorm = 1.2944, lr_0 = 9.1537e-04
Validation prc-auc = 0.301696
Epoch 60
Validation prc-auc = 0.279037
Epoch 61
Validation prc-auc = 0.307725
Epoch 62
Validation prc-auc = 0.323613
Epoch 63
Validation prc-auc = 0.291489
Epoch 64
Loss = 4.3935e-02, PNorm = 83.1453, GNorm = 0.3841, lr_0 = 9.0754e-04
Validation prc-auc = 0.237511
Epoch 65
Validation prc-auc = 0.206105
Epoch 66
Validation prc-auc = 0.213431
Epoch 67
Validation prc-auc = 0.201107
Epoch 68
Validation prc-auc = 0.205966
Epoch 69
Loss = 3.8325e-02, PNorm = 83.5038, GNorm = 0.6145, lr_0 = 8.9978e-04
Validation prc-auc = 0.210441
Epoch 70
Validation prc-auc = 0.203410
Epoch 71
Validation prc-auc = 0.230668
Epoch 72
Validation prc-auc = 0.244779
Epoch 73
Validation prc-auc = 0.196868
Epoch 74
Loss = 1.0429e-01, PNorm = 83.8143, GNorm = 3.5757, lr_0 = 8.9209e-04
Validation prc-auc = 0.252263
Epoch 75
Validation prc-auc = 0.282624
Epoch 76
Validation prc-auc = 0.206495
Epoch 77
Validation prc-auc = 0.203314
Epoch 78
Validation prc-auc = 0.246712
Epoch 79
Loss = 2.2613e-02, PNorm = 84.1355, GNorm = 1.1337, lr_0 = 8.8447e-04
Validation prc-auc = 0.296338
Epoch 80
Validation prc-auc = 0.259639
Epoch 81
Validation prc-auc = 0.227287
Epoch 82
Validation prc-auc = 0.263878
Epoch 83
Validation prc-auc = 0.265886
Epoch 84
Loss = 1.8350e-02, PNorm = 84.4276, GNorm = 0.6847, lr_0 = 8.7691e-04
Validation prc-auc = 0.221338
Epoch 85
Validation prc-auc = 0.228542
Epoch 86
Validation prc-auc = 0.236924
Epoch 87
Validation prc-auc = 0.244105
Epoch 88
Validation prc-auc = 0.234104
Epoch 89
Loss = 4.6554e-02, PNorm = 84.6735, GNorm = 0.5354, lr_0 = 8.6941e-04
Validation prc-auc = 0.192770
Epoch 90
Validation prc-auc = 0.203480
Epoch 91
Validation prc-auc = 0.241460
Epoch 92
Validation prc-auc = 0.254533
Epoch 93
Validation prc-auc = 0.243593
Epoch 94
Loss = 3.3112e-02, PNorm = 84.9733, GNorm = 0.6421, lr_0 = 8.6198e-04
Validation prc-auc = 0.230764
Epoch 95
Validation prc-auc = 0.230331
Epoch 96
Validation prc-auc = 0.205606
Epoch 97
Validation prc-auc = 0.178229
Epoch 98
Validation prc-auc = 0.182735
Epoch 99
Loss = 1.0864e-02, PNorm = 85.2441, GNorm = 0.1917, lr_0 = 8.5461e-04
Validation prc-auc = 0.213917
Epoch 100
Validation prc-auc = 0.239236
Epoch 101
Validation prc-auc = 0.281919
Epoch 102
Validation prc-auc = 0.287679
Epoch 103
Validation prc-auc = 0.255842
Epoch 104
Loss = 9.8627e-03, PNorm = 85.4615, GNorm = 0.2517, lr_0 = 8.4730e-04
Validation prc-auc = 0.231141
Epoch 105
Validation prc-auc = 0.227071
Epoch 106
Validation prc-auc = 0.257809
Epoch 107
Validation prc-auc = 0.267455
Epoch 108
Validation prc-auc = 0.277898
Epoch 109
Loss = 4.2696e-03, PNorm = 85.6242, GNorm = 0.1194, lr_0 = 8.4006e-04
Validation prc-auc = 0.281482
Epoch 110
Validation prc-auc = 0.328216
Epoch 111
Validation prc-auc = 0.310008
Epoch 112
Validation prc-auc = 0.267946
Epoch 113
Validation prc-auc = 0.247763
Epoch 114
Loss = 1.0243e-02, PNorm = 85.7935, GNorm = 0.4677, lr_0 = 8.3288e-04
Validation prc-auc = 0.261461
Epoch 115
Validation prc-auc = 0.242210
Epoch 116
Validation prc-auc = 0.242133
Epoch 117
Validation prc-auc = 0.276419
Epoch 118
Validation prc-auc = 0.200080
Epoch 119
Loss = 7.4002e-02, PNorm = 86.1586, GNorm = 2.8840, lr_0 = 8.2576e-04
Validation prc-auc = 0.180384
Epoch 120
Validation prc-auc = 0.285562
Epoch 121
Validation prc-auc = 0.311209
Epoch 122
Validation prc-auc = 0.294940
Epoch 123
Validation prc-auc = 0.249885
Epoch 124
Loss = 1.7930e-01, PNorm = 86.5154, GNorm = 0.8035, lr_0 = 8.1870e-04
Validation prc-auc = 0.255555
Epoch 125
Validation prc-auc = 0.230887
Epoch 126
Validation prc-auc = 0.228909
Epoch 127
Validation prc-auc = 0.264813
Epoch 128
Validation prc-auc = 0.275429
Epoch 129
Loss = 1.8357e-01, PNorm = 86.8617, GNorm = 4.5801, lr_0 = 8.1170e-04
Validation prc-auc = 0.229094
Epoch 130
Validation prc-auc = 0.194775
Epoch 131
Validation prc-auc = 0.168031
Epoch 132
Validation prc-auc = 0.191172
Epoch 133
Validation prc-auc = 0.227478
Epoch 134
Loss = 1.1342e-02, PNorm = 87.2355, GNorm = 0.2556, lr_0 = 8.0476e-04
Validation prc-auc = 0.246038
Epoch 135
Validation prc-auc = 0.255616
Epoch 136
Validation prc-auc = 0.254731
Epoch 137
Validation prc-auc = 0.249726
Epoch 138
Validation prc-auc = 0.245840
Epoch 139
Loss = 6.2263e-03, PNorm = 87.4927, GNorm = 0.0195, lr_0 = 7.9788e-04
Validation prc-auc = 0.244570
Epoch 140
Validation prc-auc = 0.235615
Epoch 141
Validation prc-auc = 0.220272
Epoch 142
Validation prc-auc = 0.208459
Epoch 143
Validation prc-auc = 0.200123
Epoch 144
Loss = 4.0279e-03, PNorm = 87.6342, GNorm = 0.1075, lr_0 = 7.9106e-04
Validation prc-auc = 0.192568
Epoch 145
Validation prc-auc = 0.186821
Epoch 146
Validation prc-auc = 0.184707
Epoch 147
Validation prc-auc = 0.195172
Epoch 148
Validation prc-auc = 0.213246
Epoch 149
Loss = 3.8667e-03, PNorm = 87.7297, GNorm = 0.1326, lr_0 = 7.8430e-04
Validation prc-auc = 0.215438
Epoch 150
Validation prc-auc = 0.202191
Epoch 151
Validation prc-auc = 0.202999
Epoch 152
Validation prc-auc = 0.195512
Epoch 153
Validation prc-auc = 0.212758
Epoch 154
Loss = 3.2520e-03, PNorm = 87.8064, GNorm = 0.0373, lr_0 = 7.7759e-04
Validation prc-auc = 0.237215
Epoch 155
Validation prc-auc = 0.244448
Epoch 156
Validation prc-auc = 0.251906
Epoch 157
Validation prc-auc = 0.237463
Epoch 158
Validation prc-auc = 0.221265
Epoch 159
Loss = 1.5139e-03, PNorm = 87.8657, GNorm = 0.0233, lr_0 = 7.7095e-04
Validation prc-auc = 0.219616
Epoch 160
Validation prc-auc = 0.210079
Epoch 161
Validation prc-auc = 0.208258
Epoch 162
Validation prc-auc = 0.207564
Epoch 163
Validation prc-auc = 0.211004
Epoch 164
Loss = 2.6689e-03, PNorm = 87.9153, GNorm = 0.0170, lr_0 = 7.6436e-04
Validation prc-auc = 0.215004
Epoch 165
Validation prc-auc = 0.227429
Epoch 166
Validation prc-auc = 0.244716
Epoch 167
Validation prc-auc = 0.243009
Epoch 168
Validation prc-auc = 0.246214
Epoch 169
Loss = 1.8959e-03, PNorm = 87.9682, GNorm = 0.0144, lr_0 = 7.5782e-04
Validation prc-auc = 0.244058
Epoch 170
Validation prc-auc = 0.246435
Epoch 171
Validation prc-auc = 0.237189
Epoch 172
Validation prc-auc = 0.231434
Epoch 173
Validation prc-auc = 0.230957
Epoch 174
Loss = 3.2964e-03, PNorm = 88.0205, GNorm = 0.2178, lr_0 = 7.5134e-04
Validation prc-auc = 0.230271
Epoch 175
Validation prc-auc = 0.236124
Epoch 176
Validation prc-auc = 0.234886
Epoch 177
Validation prc-auc = 0.233873
Epoch 178
Validation prc-auc = 0.238762
Epoch 179
Loss = 9.4037e-04, PNorm = 88.0521, GNorm = 0.0239, lr_0 = 7.4492e-04
Validation prc-auc = 0.240527
Epoch 180
Validation prc-auc = 0.241138
Epoch 181
Validation prc-auc = 0.238063
Epoch 182
Validation prc-auc = 0.238238
Epoch 183
Validation prc-auc = 0.234292
Epoch 184
Loss = 4.1477e-04, PNorm = 88.0769, GNorm = 0.0094, lr_0 = 7.3855e-04
Validation prc-auc = 0.234197
Epoch 185
Validation prc-auc = 0.234250
Epoch 186
Validation prc-auc = 0.233476
Epoch 187
Validation prc-auc = 0.233148
Epoch 188
Validation prc-auc = 0.233055
Epoch 189
Loss = 6.4001e-04, PNorm = 88.0948, GNorm = 0.0178, lr_0 = 7.3224e-04
Validation prc-auc = 0.232867
Epoch 190
Validation prc-auc = 0.233167
Epoch 191
Validation prc-auc = 0.233055
Epoch 192
Validation prc-auc = 0.236434
Epoch 193
Validation prc-auc = 0.236434
Epoch 194
Loss = 3.1285e-04, PNorm = 88.1079, GNorm = 0.0107, lr_0 = 7.2598e-04
Validation prc-auc = 0.236260
Epoch 195
Validation prc-auc = 0.235655
Epoch 196
Validation prc-auc = 0.236170
Epoch 197
Validation prc-auc = 0.239206
Epoch 198
Validation prc-auc = 0.239687
Epoch 199
Loss = 1.6953e-04, PNorm = 88.1196, GNorm = 0.0048, lr_0 = 7.1977e-04
Validation prc-auc = 0.244501
Epoch 200
Validation prc-auc = 0.244501
Epoch 201
Validation prc-auc = 0.244448
Epoch 202
Validation prc-auc = 0.244448
Epoch 203
Validation prc-auc = 0.244448
Epoch 204
Loss = 1.7062e-04, PNorm = 88.1297, GNorm = 0.0036, lr_0 = 7.1362e-04
Validation prc-auc = 0.248064
Epoch 205
Validation prc-auc = 0.248064
Epoch 206
Validation prc-auc = 0.248155
Epoch 207
Validation prc-auc = 0.243041
Epoch 208
Validation prc-auc = 0.243115
Epoch 209
Loss = 3.0831e-04, PNorm = 88.1389, GNorm = 0.0104, lr_0 = 7.0752e-04
Validation prc-auc = 0.243065
Epoch 210
Validation prc-auc = 0.243065
Epoch 211
Validation prc-auc = 0.242946
Epoch 212
Validation prc-auc = 0.242946
Epoch 213
Validation prc-auc = 0.243838
Epoch 214
Loss = 3.3854e-04, PNorm = 88.1500, GNorm = 0.0195, lr_0 = 7.0147e-04
Validation prc-auc = 0.243941
Epoch 215
Validation prc-auc = 0.243829
Epoch 216
Validation prc-auc = 0.243689
Epoch 217
Validation prc-auc = 0.243780
Epoch 218
Validation prc-auc = 0.243670
Epoch 219
Loss = 1.2852e-04, PNorm = 88.1621, GNorm = 0.0031, lr_0 = 6.9548e-04
Validation prc-auc = 0.243670
Epoch 220
Validation prc-auc = 0.243670
Epoch 221
Validation prc-auc = 0.243344
Epoch 222
Validation prc-auc = 0.243896
Epoch 223
Validation prc-auc = 0.247709
Epoch 224
Loss = 9.1210e-05, PNorm = 88.1722, GNorm = 0.0038, lr_0 = 6.8953e-04
Validation prc-auc = 0.247617
Epoch 225
Validation prc-auc = 0.247228
Epoch 226
Validation prc-auc = 0.247228
Epoch 227
Validation prc-auc = 0.247391
Epoch 228
Validation prc-auc = 0.247207
Epoch 229
Loss = 1.6087e-04, PNorm = 88.1795, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.247207
Epoch 230
Validation prc-auc = 0.243668
Epoch 231
Validation prc-auc = 0.243751
Epoch 232
Validation prc-auc = 0.243055
Epoch 233
Validation prc-auc = 0.243023
Epoch 234
Loss = 3.0499e-04, PNorm = 88.1857, GNorm = 0.0040, lr_0 = 6.7779e-04
Validation prc-auc = 0.242620
Epoch 235
Validation prc-auc = 0.242620
Epoch 236
Validation prc-auc = 0.246080
Epoch 237
Validation prc-auc = 0.245971
Epoch 238
Validation prc-auc = 0.246027
Epoch 239
Loss = 1.0634e-04, PNorm = 88.1956, GNorm = 0.0032, lr_0 = 6.7200e-04
Validation prc-auc = 0.246119
Epoch 240
Validation prc-auc = 0.246074
Epoch 241
Validation prc-auc = 0.246182
Epoch 242
Validation prc-auc = 0.246800
Epoch 243
Validation prc-auc = 0.246733
Epoch 244
Loss = 7.6656e-05, PNorm = 88.2041, GNorm = 0.0031, lr_0 = 6.6625e-04
Validation prc-auc = 0.246733
Epoch 245
Validation prc-auc = 0.250039
Epoch 246
Validation prc-auc = 0.250039
Epoch 247
Validation prc-auc = 0.250039
Epoch 248
Validation prc-auc = 0.252690
Epoch 249
Loss = 8.8182e-05, PNorm = 88.2116, GNorm = 0.0031, lr_0 = 6.6056e-04
Validation prc-auc = 0.252536
Epoch 250
Validation prc-auc = 0.252603
Epoch 251
Validation prc-auc = 0.252820
Epoch 252
Validation prc-auc = 0.252596
Epoch 253
Validation prc-auc = 0.252877
Epoch 254
Loss = 1.1104e-04, PNorm = 88.2205, GNorm = 0.0021, lr_0 = 6.5491e-04
Validation prc-auc = 0.252811
Epoch 255
Validation prc-auc = 0.252208
Epoch 256
Validation prc-auc = 0.252420
Epoch 257
Validation prc-auc = 0.251804
Epoch 258
Validation prc-auc = 0.251741
Epoch 259
Loss = 8.3043e-05, PNorm = 88.2293, GNorm = 0.0040, lr_0 = 6.4931e-04
Validation prc-auc = 0.251280
Epoch 260
Validation prc-auc = 0.250420
Epoch 261
Validation prc-auc = 0.250420
Epoch 262
Validation prc-auc = 0.250327
Epoch 263
Validation prc-auc = 0.250420
Epoch 264
Loss = 4.5222e-05, PNorm = 88.2363, GNorm = 0.0010, lr_0 = 6.4376e-04
Validation prc-auc = 0.250901
Epoch 265
Validation prc-auc = 0.251021
Epoch 266
Validation prc-auc = 0.250712
Epoch 267
Validation prc-auc = 0.250231
Epoch 268
Validation prc-auc = 0.250298
Epoch 269
Loss = 4.5276e-05, PNorm = 88.2420, GNorm = 0.0015, lr_0 = 6.3826e-04
Validation prc-auc = 0.254069
Epoch 270
Validation prc-auc = 0.253805
Epoch 271
Validation prc-auc = 0.253968
Epoch 272
Validation prc-auc = 0.253546
Epoch 273
Validation prc-auc = 0.253968
Epoch 274
Loss = 5.4851e-05, PNorm = 88.2473, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.254352
Epoch 275
Validation prc-auc = 0.254282
Epoch 276
Validation prc-auc = 0.254763
Epoch 277
Validation prc-auc = 0.254830
Epoch 278
Validation prc-auc = 0.254807
Epoch 279
Loss = 3.4437e-05, PNorm = 88.2520, GNorm = 0.0017, lr_0 = 6.2739e-04
Validation prc-auc = 0.255005
Epoch 280
Validation prc-auc = 0.255532
Epoch 281
Validation prc-auc = 0.255532
Epoch 282
Validation prc-auc = 0.255348
Epoch 283
Validation prc-auc = 0.255501
Epoch 284
Loss = 6.7237e-05, PNorm = 88.2572, GNorm = 0.0009, lr_0 = 6.2203e-04
Validation prc-auc = 0.255957
Epoch 285
Validation prc-auc = 0.256053
Epoch 286
Validation prc-auc = 0.260197
Epoch 287
Validation prc-auc = 0.260197
Epoch 288
Validation prc-auc = 0.266863
Epoch 289
Loss = 2.3074e-05, PNorm = 88.2622, GNorm = 0.0009, lr_0 = 6.1671e-04
Validation prc-auc = 0.266264
Epoch 290
Validation prc-auc = 0.266060
Epoch 291
Validation prc-auc = 0.266046
Epoch 292
Validation prc-auc = 0.265635
Epoch 293
Validation prc-auc = 0.265446
Epoch 294
Loss = 3.0786e-05, PNorm = 88.2662, GNorm = 0.0002, lr_0 = 6.1144e-04
Validation prc-auc = 0.265517
Epoch 295
Validation prc-auc = 0.265998
Epoch 296
Validation prc-auc = 0.266091
Epoch 297
Validation prc-auc = 0.266091
Epoch 298
Validation prc-auc = 0.266174
Epoch 299
Loss = 3.7240e-05, PNorm = 88.2701, GNorm = 0.0037, lr_0 = 6.0621e-04
Validation prc-auc = 0.266137
Model 0 best validation prc-auc = 0.487608 on epoch 16
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.528662
Ensemble test prc-auc = 0.528662
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.420661
Epoch 1
Validation prc-auc = 0.457936
Epoch 2
Validation prc-auc = 0.458228
Epoch 3
Validation prc-auc = 0.461219
Epoch 4
Loss = 6.7879e-01, PNorm = 78.5484, GNorm = 9.4250, lr_0 = 6.5000e-04
Validation prc-auc = 0.457434
Epoch 5
Validation prc-auc = 0.485822
Epoch 6
Validation prc-auc = 0.469495
Epoch 7
Validation prc-auc = 0.474223
Epoch 8
Validation prc-auc = 0.490912
Epoch 9
Loss = 6.7939e-01, PNorm = 78.7767, GNorm = 12.4080, lr_0 = 9.9743e-04
Validation prc-auc = 0.337378
Epoch 10
Validation prc-auc = 0.301258
Epoch 11
Validation prc-auc = 0.259962
Epoch 12
Validation prc-auc = 0.370990
Epoch 13
Validation prc-auc = 0.403101
Epoch 14
Loss = 6.1304e-01, PNorm = 79.0637, GNorm = 0.8893, lr_0 = 9.8890e-04
Validation prc-auc = 0.309827
Epoch 15
Validation prc-auc = 0.291735
Epoch 16
Validation prc-auc = 0.307380
Epoch 17
Validation prc-auc = 0.308140
Epoch 18
Validation prc-auc = 0.428703
Epoch 19
Loss = 5.2677e-01, PNorm = 79.3599, GNorm = 2.3717, lr_0 = 9.8045e-04
Validation prc-auc = 0.404927
Epoch 20
Validation prc-auc = 0.361680
Epoch 21
Validation prc-auc = 0.450644
Epoch 22
Validation prc-auc = 0.426842
Epoch 23
Validation prc-auc = 0.392685
Epoch 24
Loss = 4.6043e-01, PNorm = 79.6595, GNorm = 4.8828, lr_0 = 9.7207e-04
Validation prc-auc = 0.349631
Epoch 25
Validation prc-auc = 0.326684
Epoch 26
Validation prc-auc = 0.319028
Epoch 27
Validation prc-auc = 0.348372
Epoch 28
Validation prc-auc = 0.288309
Epoch 29
Loss = 3.7091e-01, PNorm = 79.9930, GNorm = 1.4221, lr_0 = 9.6376e-04
Validation prc-auc = 0.297019
Epoch 30
Validation prc-auc = 0.285389
Epoch 31
Validation prc-auc = 0.314174
Epoch 32
Validation prc-auc = 0.328227
Epoch 33
Validation prc-auc = 0.425654
Epoch 34
Loss = 3.2539e-01, PNorm = 80.3645, GNorm = 2.5136, lr_0 = 9.5552e-04
Validation prc-auc = 0.451703
Epoch 35
Validation prc-auc = 0.439275
Epoch 36
Validation prc-auc = 0.344190
Epoch 37
Validation prc-auc = 0.233892
Epoch 38
Validation prc-auc = 0.332029
Epoch 39
Loss = 2.5436e-01, PNorm = 80.7576, GNorm = 4.8816, lr_0 = 9.4735e-04
Validation prc-auc = 0.360506
Epoch 40
Validation prc-auc = 0.365633
Epoch 41
Validation prc-auc = 0.363018
Epoch 42
Validation prc-auc = 0.362571
Epoch 43
Validation prc-auc = 0.280235
Epoch 44
Loss = 2.9132e-01, PNorm = 81.2063, GNorm = 4.9521, lr_0 = 9.3925e-04
Validation prc-auc = 0.313984
Epoch 45
Validation prc-auc = 0.341950
Epoch 46
Validation prc-auc = 0.338457
Epoch 47
Validation prc-auc = 0.309914
Epoch 48
Validation prc-auc = 0.190773
Epoch 49
Loss = 2.0441e-01, PNorm = 81.7002, GNorm = 1.1314, lr_0 = 9.3122e-04
Validation prc-auc = 0.162704
Epoch 50
Validation prc-auc = 0.248556
Epoch 51
Validation prc-auc = 0.297140
Epoch 52
Validation prc-auc = 0.301968
Epoch 53
Validation prc-auc = 0.312886
Epoch 54
Loss = 1.0181e-01, PNorm = 82.1900, GNorm = 0.4953, lr_0 = 9.2326e-04
Validation prc-auc = 0.186690
Epoch 55
Validation prc-auc = 0.328122
Epoch 56
Validation prc-auc = 0.225372
Epoch 57
Validation prc-auc = 0.280809
Epoch 58
Validation prc-auc = 0.329574
Epoch 59
Loss = 3.3973e-01, PNorm = 82.6580, GNorm = 4.7337, lr_0 = 9.1537e-04
Validation prc-auc = 0.346119
Epoch 60
Validation prc-auc = 0.279217
Epoch 61
Validation prc-auc = 0.288589
Epoch 62
Validation prc-auc = 0.334008
Epoch 63
Validation prc-auc = 0.298605
Epoch 64
Loss = 8.9413e-02, PNorm = 83.0737, GNorm = 0.3650, lr_0 = 9.0754e-04
Validation prc-auc = 0.307533
Epoch 65
Validation prc-auc = 0.287666
Epoch 66
Validation prc-auc = 0.270246
Epoch 67
Validation prc-auc = 0.226745
Epoch 68
Validation prc-auc = 0.221687
Epoch 69
Loss = 7.1962e-02, PNorm = 83.4600, GNorm = 1.0944, lr_0 = 8.9978e-04
Validation prc-auc = 0.226417
Epoch 70
Validation prc-auc = 0.256125
Epoch 71
Validation prc-auc = 0.272778
Epoch 72
Validation prc-auc = 0.247839
Epoch 73
Validation prc-auc = 0.248686
Epoch 74
Loss = 6.4427e-02, PNorm = 83.7716, GNorm = 0.3512, lr_0 = 8.9209e-04
Validation prc-auc = 0.311930
Epoch 75
Validation prc-auc = 0.320713
Epoch 76
Validation prc-auc = 0.306418
Epoch 77
Validation prc-auc = 0.293195
Epoch 78
Validation prc-auc = 0.283546
Epoch 79
Loss = 5.7602e-02, PNorm = 84.0756, GNorm = 0.7084, lr_0 = 8.8447e-04
Validation prc-auc = 0.250902
Epoch 80
Validation prc-auc = 0.233777
Epoch 81
Validation prc-auc = 0.224770
Epoch 82
Validation prc-auc = 0.240630
Epoch 83
Validation prc-auc = 0.257950
Epoch 84
Loss = 2.8243e-02, PNorm = 84.3772, GNorm = 0.0990, lr_0 = 8.7691e-04
Validation prc-auc = 0.239297
Epoch 85
Validation prc-auc = 0.275845
Epoch 86
Validation prc-auc = 0.296512
Epoch 87
Validation prc-auc = 0.275751
Epoch 88
Validation prc-auc = 0.281087
Epoch 89
Loss = 1.3489e-02, PNorm = 84.6147, GNorm = 0.8700, lr_0 = 8.6941e-04
Validation prc-auc = 0.300239
Epoch 90
Validation prc-auc = 0.307408
Epoch 91
Validation prc-auc = 0.305317
Epoch 92
Validation prc-auc = 0.283052
Epoch 93
Validation prc-auc = 0.263338
Epoch 94
Loss = 4.1247e-02, PNorm = 84.8298, GNorm = 0.7225, lr_0 = 8.6198e-04
Validation prc-auc = 0.242708
Epoch 95
Validation prc-auc = 0.217276
Epoch 96
Validation prc-auc = 0.243376
Epoch 97
Validation prc-auc = 0.294436
Epoch 98
Validation prc-auc = 0.307211
Epoch 99
Loss = 1.7993e-02, PNorm = 85.0559, GNorm = 0.5066, lr_0 = 8.5461e-04
Validation prc-auc = 0.292837
Epoch 100
Validation prc-auc = 0.282350
Epoch 101
Validation prc-auc = 0.246013
Epoch 102
Validation prc-auc = 0.248986
Epoch 103
Validation prc-auc = 0.263404
Epoch 104
Loss = 3.3377e-02, PNorm = 85.2747, GNorm = 0.1569, lr_0 = 8.4730e-04
Validation prc-auc = 0.336663
Epoch 105
Validation prc-auc = 0.346621
Epoch 106
Validation prc-auc = 0.342681
Epoch 107
Validation prc-auc = 0.334539
Epoch 108
Validation prc-auc = 0.328910
Epoch 109
Loss = 2.5633e-02, PNorm = 85.4632, GNorm = 1.8693, lr_0 = 8.4006e-04
Validation prc-auc = 0.302107
Epoch 110
Validation prc-auc = 0.211599
Epoch 111
Validation prc-auc = 0.264865
Epoch 112
Validation prc-auc = 0.275362
Epoch 113
Validation prc-auc = 0.229602
Epoch 114
Loss = 6.8220e-02, PNorm = 85.6689, GNorm = 2.2773, lr_0 = 8.3288e-04
Validation prc-auc = 0.275315
Epoch 115
Validation prc-auc = 0.304039
Epoch 116
Validation prc-auc = 0.255938
Epoch 117
Validation prc-auc = 0.233755
Epoch 118
Validation prc-auc = 0.216920
Epoch 119
Loss = 6.1016e-02, PNorm = 85.9954, GNorm = 0.2323, lr_0 = 8.2576e-04
Validation prc-auc = 0.234682
Epoch 120
Validation prc-auc = 0.222558
Epoch 121
Validation prc-auc = 0.219669
Epoch 122
Validation prc-auc = 0.237908
Epoch 123
Validation prc-auc = 0.228341
Epoch 124
Loss = 2.3777e-02, PNorm = 86.2930, GNorm = 0.1455, lr_0 = 8.1870e-04
Validation prc-auc = 0.210794
Epoch 125
Validation prc-auc = 0.199932
Epoch 126
Validation prc-auc = 0.221785
Epoch 127
Validation prc-auc = 0.228188
Epoch 128
Validation prc-auc = 0.229822
Epoch 129
Loss = 2.1434e-02, PNorm = 86.5291, GNorm = 0.5616, lr_0 = 8.1170e-04
Validation prc-auc = 0.243109
Epoch 130
Validation prc-auc = 0.236189
Epoch 131
Validation prc-auc = 0.207302
Epoch 132
Validation prc-auc = 0.193973
Epoch 133
Validation prc-auc = 0.217384
Epoch 134
Loss = 1.2687e-01, PNorm = 86.7778, GNorm = 2.5584, lr_0 = 8.0476e-04
Validation prc-auc = 0.221427
Epoch 135
Validation prc-auc = 0.225822
Epoch 136
Validation prc-auc = 0.221941
Epoch 137
Validation prc-auc = 0.224318
Epoch 138
Validation prc-auc = 0.235118
Epoch 139
Loss = 1.2207e-02, PNorm = 86.9729, GNorm = 0.0883, lr_0 = 7.9788e-04
Validation prc-auc = 0.241458
Epoch 140
Validation prc-auc = 0.247542
Epoch 141
Validation prc-auc = 0.255155
Epoch 142
Validation prc-auc = 0.255968
Epoch 143
Validation prc-auc = 0.274385
Epoch 144
Loss = 3.4209e-02, PNorm = 87.1143, GNorm = 0.5921, lr_0 = 7.9106e-04
Validation prc-auc = 0.285068
Epoch 145
Validation prc-auc = 0.279809
Epoch 146
Validation prc-auc = 0.269946
Epoch 147
Validation prc-auc = 0.262443
Epoch 148
Validation prc-auc = 0.239521
Epoch 149
Loss = 7.4511e-03, PNorm = 87.2699, GNorm = 0.1724, lr_0 = 7.8430e-04
Validation prc-auc = 0.227633
Epoch 150
Validation prc-auc = 0.218389
Epoch 151
Validation prc-auc = 0.214915
Epoch 152
Validation prc-auc = 0.260338
Epoch 153
Validation prc-auc = 0.270727
Epoch 154
Loss = 4.3829e-03, PNorm = 87.4757, GNorm = 0.0986, lr_0 = 7.7759e-04
Validation prc-auc = 0.248127
Epoch 155
Validation prc-auc = 0.271496
Epoch 156
Validation prc-auc = 0.279499
Epoch 157
Validation prc-auc = 0.273700
Epoch 158
Validation prc-auc = 0.233976
Epoch 159
Loss = 3.6222e-02, PNorm = 87.6901, GNorm = 0.5669, lr_0 = 7.7095e-04
Validation prc-auc = 0.181578
Epoch 160
Validation prc-auc = 0.197170
Epoch 161
Validation prc-auc = 0.211570
Epoch 162
Validation prc-auc = 0.206212
Epoch 163
Validation prc-auc = 0.215330
Epoch 164
Loss = 1.2465e-01, PNorm = 87.9610, GNorm = 2.2373, lr_0 = 7.6436e-04
Validation prc-auc = 0.265177
Epoch 165
Validation prc-auc = 0.274934
Epoch 166
Validation prc-auc = 0.231637
Epoch 167
Validation prc-auc = 0.204012
Epoch 168
Validation prc-auc = 0.189641
Epoch 169
Loss = 3.4128e-02, PNorm = 88.2434, GNorm = 0.3778, lr_0 = 7.5782e-04
Validation prc-auc = 0.208085
Epoch 170
Validation prc-auc = 0.257314
Epoch 171
Validation prc-auc = 0.319175
Epoch 172
Validation prc-auc = 0.315239
Epoch 173
Validation prc-auc = 0.280051
Epoch 174
Loss = 6.4917e-03, PNorm = 88.5055, GNorm = 0.1110, lr_0 = 7.5134e-04
Validation prc-auc = 0.211732
Epoch 175
Validation prc-auc = 0.184104
Epoch 176
Validation prc-auc = 0.207875
Epoch 177
Validation prc-auc = 0.237450
Epoch 178
Validation prc-auc = 0.266153
Epoch 179
Loss = 6.7845e-03, PNorm = 88.6905, GNorm = 0.0565, lr_0 = 7.4492e-04
Validation prc-auc = 0.292587
Epoch 180
Validation prc-auc = 0.296856
Epoch 181
Validation prc-auc = 0.306850
Epoch 182
Validation prc-auc = 0.286291
Epoch 183
Validation prc-auc = 0.282767
Epoch 184
Loss = 3.5275e-03, PNorm = 88.8265, GNorm = 0.0604, lr_0 = 7.3855e-04
Validation prc-auc = 0.268273
Epoch 185
Validation prc-auc = 0.266838
Epoch 186
Validation prc-auc = 0.263637
Epoch 187
Validation prc-auc = 0.261352
Epoch 188
Validation prc-auc = 0.267484
Epoch 189
Loss = 2.5836e-03, PNorm = 88.9277, GNorm = 0.0224, lr_0 = 7.3224e-04
Validation prc-auc = 0.267054
Epoch 190
Validation prc-auc = 0.268399
Epoch 191
Validation prc-auc = 0.276735
Epoch 192
Validation prc-auc = 0.287725
Epoch 193
Validation prc-auc = 0.288679
Epoch 194
Loss = 1.9950e-03, PNorm = 88.9891, GNorm = 0.0316, lr_0 = 7.2598e-04
Validation prc-auc = 0.289114
Epoch 195
Validation prc-auc = 0.288929
Epoch 196
Validation prc-auc = 0.282456
Epoch 197
Validation prc-auc = 0.272728
Epoch 198
Validation prc-auc = 0.271680
Epoch 199
Loss = 8.0198e-04, PNorm = 89.0349, GNorm = 0.0207, lr_0 = 7.1977e-04
Validation prc-auc = 0.271598
Epoch 200
Validation prc-auc = 0.271760
Epoch 201
Validation prc-auc = 0.271311
Epoch 202
Validation prc-auc = 0.271315
Epoch 203
Validation prc-auc = 0.271311
Epoch 204
Loss = 1.0285e-03, PNorm = 89.0619, GNorm = 0.1913, lr_0 = 7.1362e-04
Validation prc-auc = 0.271392
Epoch 205
Validation prc-auc = 0.278023
Epoch 206
Validation prc-auc = 0.281764
Epoch 207
Validation prc-auc = 0.281656
Epoch 208
Validation prc-auc = 0.283690
Epoch 209
Loss = 1.4612e-03, PNorm = 89.0858, GNorm = 0.0122, lr_0 = 7.0752e-04
Validation prc-auc = 0.283638
Epoch 210
Validation prc-auc = 0.283437
Epoch 211
Validation prc-auc = 0.281394
Epoch 212
Validation prc-auc = 0.279675
Epoch 213
Validation prc-auc = 0.279458
Epoch 214
Loss = 4.5339e-04, PNorm = 89.1154, GNorm = 0.0102, lr_0 = 7.0147e-04
Validation prc-auc = 0.279104
Epoch 215
Validation prc-auc = 0.279294
Epoch 216
Validation prc-auc = 0.277534
Epoch 217
Validation prc-auc = 0.277420
Epoch 218
Validation prc-auc = 0.277356
Epoch 219
Loss = 1.2988e-03, PNorm = 89.1366, GNorm = 0.0905, lr_0 = 6.9548e-04
Validation prc-auc = 0.277290
Epoch 220
Validation prc-auc = 0.277382
Epoch 221
Validation prc-auc = 0.277386
Epoch 222
Validation prc-auc = 0.277469
Epoch 223
Validation prc-auc = 0.277310
Epoch 224
Loss = 2.4854e-04, PNorm = 89.1491, GNorm = 0.0055, lr_0 = 6.8953e-04
Validation prc-auc = 0.277302
Epoch 225
Validation prc-auc = 0.277302
Epoch 226
Validation prc-auc = 0.277252
Epoch 227
Validation prc-auc = 0.277247
Epoch 228
Validation prc-auc = 0.277420
Epoch 229
Loss = 1.6646e-04, PNorm = 89.1637, GNorm = 0.0050, lr_0 = 6.8364e-04
Validation prc-auc = 0.277234
Epoch 230
Validation prc-auc = 0.277172
Epoch 231
Validation prc-auc = 0.277155
Epoch 232
Validation prc-auc = 0.277040
Epoch 233
Validation prc-auc = 0.283128
Epoch 234
Loss = 1.7735e-04, PNorm = 89.1747, GNorm = 0.0067, lr_0 = 6.7779e-04
Validation prc-auc = 0.283067
Epoch 235
Validation prc-auc = 0.283079
Epoch 236
Validation prc-auc = 0.283030
Epoch 237
Validation prc-auc = 0.281935
Epoch 238
Validation prc-auc = 0.281935
Epoch 239
Loss = 1.2403e-04, PNorm = 89.1835, GNorm = 0.0077, lr_0 = 6.7200e-04
Validation prc-auc = 0.281935
Epoch 240
Validation prc-auc = 0.281935
Epoch 241
Validation prc-auc = 0.281935
Epoch 242
Validation prc-auc = 0.282439
Epoch 243
Validation prc-auc = 0.282439
Epoch 244
Loss = 1.6262e-04, PNorm = 89.1912, GNorm = 0.0020, lr_0 = 6.6625e-04
Validation prc-auc = 0.282439
Epoch 245
Validation prc-auc = 0.282390
Epoch 246
Validation prc-auc = 0.282390
Epoch 247
Validation prc-auc = 0.282342
Epoch 248
Validation prc-auc = 0.282403
Epoch 249
Loss = 1.6689e-04, PNorm = 89.1992, GNorm = 0.0021, lr_0 = 6.6056e-04
Validation prc-auc = 0.282403
Epoch 250
Validation prc-auc = 0.282470
Epoch 251
Validation prc-auc = 0.282424
Epoch 252
Validation prc-auc = 0.282487
Epoch 253
Validation prc-auc = 0.282541
Epoch 254
Loss = 1.0802e-04, PNorm = 89.2060, GNorm = 0.0025, lr_0 = 6.5491e-04
Validation prc-auc = 0.282478
Epoch 255
Validation prc-auc = 0.282478
Epoch 256
Validation prc-auc = 0.282416
Epoch 257
Validation prc-auc = 0.282416
Epoch 258
Validation prc-auc = 0.282355
Epoch 259
Loss = 8.0309e-05, PNorm = 89.2132, GNorm = 0.0017, lr_0 = 6.4931e-04
Validation prc-auc = 0.282355
Epoch 260
Validation prc-auc = 0.282355
Epoch 261
Validation prc-auc = 0.282948
Epoch 262
Validation prc-auc = 0.282948
Epoch 263
Validation prc-auc = 0.282948
Epoch 264
Loss = 1.2134e-04, PNorm = 89.2205, GNorm = 0.0028, lr_0 = 6.4376e-04
Validation prc-auc = 0.283067
Epoch 265
Validation prc-auc = 0.283067
Epoch 266
Validation prc-auc = 0.283067
Epoch 267
Validation prc-auc = 0.283067
Epoch 268
Validation prc-auc = 0.283067
Epoch 269
Loss = 1.1993e-04, PNorm = 89.2285, GNorm = 0.0048, lr_0 = 6.3826e-04
Validation prc-auc = 0.283122
Epoch 270
Validation prc-auc = 0.283122
Epoch 271
Validation prc-auc = 0.283183
Epoch 272
Validation prc-auc = 0.283183
Epoch 273
Validation prc-auc = 0.283183
Epoch 274
Loss = 5.0126e-05, PNorm = 89.2354, GNorm = 0.0021, lr_0 = 6.3280e-04
Validation prc-auc = 0.282589
Epoch 275
Validation prc-auc = 0.282635
Epoch 276
Validation prc-auc = 0.282635
Epoch 277
Validation prc-auc = 0.282635
Epoch 278
Validation prc-auc = 0.282589
Epoch 279
Loss = 5.5134e-05, PNorm = 89.2421, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation prc-auc = 0.282711
Epoch 280
Validation prc-auc = 0.282711
Epoch 281
Validation prc-auc = 0.282711
Epoch 282
Validation prc-auc = 0.282711
Epoch 283
Validation prc-auc = 0.282760
Epoch 284
Loss = 5.5362e-05, PNorm = 89.2500, GNorm = 0.0021, lr_0 = 6.2203e-04
Validation prc-auc = 0.282760
Epoch 285
Validation prc-auc = 0.282822
Epoch 286
Validation prc-auc = 0.282868
Epoch 287
Validation prc-auc = 0.282822
Epoch 288
Validation prc-auc = 0.283000
Epoch 289
Loss = 8.0032e-05, PNorm = 89.2569, GNorm = 0.0035, lr_0 = 6.1671e-04
Validation prc-auc = 0.283046
Epoch 290
Validation prc-auc = 0.282986
Epoch 291
Validation prc-auc = 0.283042
Epoch 292
Validation prc-auc = 0.283172
Epoch 293
Validation prc-auc = 0.283172
Epoch 294
Loss = 7.5541e-05, PNorm = 89.2602, GNorm = 0.0012, lr_0 = 6.1144e-04
Validation prc-auc = 0.283532
Epoch 295
Validation prc-auc = 0.283470
Epoch 296
Validation prc-auc = 0.283409
Epoch 297
Validation prc-auc = 0.283409
Epoch 298
Validation prc-auc = 0.283409
Epoch 299
Loss = 6.5809e-05, PNorm = 89.2681, GNorm = 0.0023, lr_0 = 6.0621e-04
Validation prc-auc = 0.283350
Model 0 best validation prc-auc = 0.490912 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.426883
Ensemble test prc-auc = 0.426883
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.450353
Epoch 1
Validation prc-auc = 0.443115
Epoch 2
Validation prc-auc = 0.453604
Epoch 3
Validation prc-auc = 0.464479
Epoch 4
Loss = 6.7479e-01, PNorm = 78.5519, GNorm = 9.3207, lr_0 = 6.5000e-04
Validation prc-auc = 0.438051
Epoch 5
Validation prc-auc = 0.437230
Epoch 6
Validation prc-auc = 0.488131
Epoch 7
Validation prc-auc = 0.433817
Epoch 8
Validation prc-auc = 0.490877
Epoch 9
Loss = 5.3238e-01, PNorm = 78.7874, GNorm = 4.2839, lr_0 = 9.9743e-04
Validation prc-auc = 0.451423
Epoch 10
Validation prc-auc = 0.447778
Epoch 11
Validation prc-auc = 0.250358
Epoch 12
Validation prc-auc = 0.251147
Epoch 13
Validation prc-auc = 0.293715
Epoch 14
Loss = 5.4735e-01, PNorm = 79.0682, GNorm = 4.1783, lr_0 = 9.8890e-04
Validation prc-auc = 0.430211
Epoch 15
Validation prc-auc = 0.442787
Epoch 16
Validation prc-auc = 0.435221
Epoch 17
Validation prc-auc = 0.442071
Epoch 18
Validation prc-auc = 0.437712
Epoch 19
Loss = 5.9960e-01, PNorm = 79.3356, GNorm = 5.0889, lr_0 = 9.8045e-04
Validation prc-auc = 0.405947
Epoch 20
Validation prc-auc = 0.415426
Epoch 21
Validation prc-auc = 0.398738
Epoch 22
Validation prc-auc = 0.191386
Epoch 23
Validation prc-auc = 0.332591
Epoch 24
Loss = 5.0185e-01, PNorm = 79.6336, GNorm = 1.1886, lr_0 = 9.7207e-04
Validation prc-auc = 0.352359
Epoch 25
Validation prc-auc = 0.346060
Epoch 26
Validation prc-auc = 0.354681
Epoch 27
Validation prc-auc = 0.374514
Epoch 28
Validation prc-auc = 0.359520
Epoch 29
Loss = 3.9279e-01, PNorm = 79.9565, GNorm = 3.0733, lr_0 = 9.6376e-04
Validation prc-auc = 0.394954
Epoch 30
Validation prc-auc = 0.378815
Epoch 31
Validation prc-auc = 0.363490
Epoch 32
Validation prc-auc = 0.284182
Epoch 33
Validation prc-auc = 0.335491
Epoch 34
Loss = 4.0500e-01, PNorm = 80.2851, GNorm = 7.8650, lr_0 = 9.5552e-04
Validation prc-auc = 0.349869
Epoch 35
Validation prc-auc = 0.369388
Epoch 36
Validation prc-auc = 0.391851
Epoch 37
Validation prc-auc = 0.375983
Epoch 38
Validation prc-auc = 0.399716
Epoch 39
Loss = 3.6006e-01, PNorm = 80.6638, GNorm = 2.7468, lr_0 = 9.4735e-04
Validation prc-auc = 0.382586
Epoch 40
Validation prc-auc = 0.383708
Epoch 41
Validation prc-auc = 0.379438
Epoch 42
Validation prc-auc = 0.344584
Epoch 43
Validation prc-auc = 0.356027
Epoch 44
Loss = 3.3708e-01, PNorm = 81.0584, GNorm = 1.1960, lr_0 = 9.3925e-04
Validation prc-auc = 0.356214
Epoch 45
Validation prc-auc = 0.333767
Epoch 46
Validation prc-auc = 0.313628
Epoch 47
Validation prc-auc = 0.336772
Epoch 48
Validation prc-auc = 0.330730
Epoch 49
Loss = 1.9644e-01, PNorm = 81.4938, GNorm = 3.2408, lr_0 = 9.3122e-04
Validation prc-auc = 0.354519
Epoch 50
Validation prc-auc = 0.351713
Epoch 51
Validation prc-auc = 0.354549
Epoch 52
Validation prc-auc = 0.319716
Epoch 53
Validation prc-auc = 0.306216
Epoch 54
Loss = 1.4992e-01, PNorm = 81.9167, GNorm = 0.7446, lr_0 = 9.2326e-04
Validation prc-auc = 0.327580
Epoch 55
Validation prc-auc = 0.325589
Epoch 56
Validation prc-auc = 0.352842
Epoch 57
Validation prc-auc = 0.354549
Epoch 58
Validation prc-auc = 0.348195
Epoch 59
Loss = 1.6586e-01, PNorm = 82.3132, GNorm = 1.7308, lr_0 = 9.1537e-04
Validation prc-auc = 0.305380
Epoch 60
Validation prc-auc = 0.332003
Epoch 61
Validation prc-auc = 0.352183
Epoch 62
Validation prc-auc = 0.269207
Epoch 63
Validation prc-auc = 0.257248
Epoch 64
Loss = 1.8356e-01, PNorm = 82.7234, GNorm = 6.7183, lr_0 = 9.0754e-04
Validation prc-auc = 0.353868
Epoch 65
Validation prc-auc = 0.302649
Epoch 66
Validation prc-auc = 0.318274
Epoch 67
Validation prc-auc = 0.329895
Epoch 68
Validation prc-auc = 0.313174
Epoch 69
Loss = 1.1830e-01, PNorm = 83.1548, GNorm = 0.8571, lr_0 = 8.9978e-04
Validation prc-auc = 0.253649
Epoch 70
Validation prc-auc = 0.259100
Epoch 71
Validation prc-auc = 0.328664
Epoch 72
Validation prc-auc = 0.267724
Epoch 73
Validation prc-auc = 0.286050
Epoch 74
Loss = 6.8920e-02, PNorm = 83.5520, GNorm = 1.7007, lr_0 = 8.9209e-04
Validation prc-auc = 0.352845
Epoch 75
Validation prc-auc = 0.351592
Epoch 76
Validation prc-auc = 0.323956
Epoch 77
Validation prc-auc = 0.292979
Epoch 78
Validation prc-auc = 0.285621
Epoch 79
Loss = 2.5831e-02, PNorm = 83.8893, GNorm = 0.1329, lr_0 = 8.8447e-04
Validation prc-auc = 0.292541
Epoch 80
Validation prc-auc = 0.287925
Epoch 81
Validation prc-auc = 0.297752
Epoch 82
Validation prc-auc = 0.307407
Epoch 83
Validation prc-auc = 0.260515
Epoch 84
Loss = 2.5320e-02, PNorm = 84.1755, GNorm = 0.3427, lr_0 = 8.7691e-04
Validation prc-auc = 0.189323
Epoch 85
Validation prc-auc = 0.225363
Epoch 86
Validation prc-auc = 0.286895
Epoch 87
Validation prc-auc = 0.292944
Epoch 88
Validation prc-auc = 0.292704
Epoch 89
Loss = 2.5646e-01, PNorm = 84.4516, GNorm = 2.0814, lr_0 = 8.6941e-04
Validation prc-auc = 0.360888
Epoch 90
Validation prc-auc = 0.360951
Epoch 91
Validation prc-auc = 0.348303
Epoch 92
Validation prc-auc = 0.320626
Epoch 93
Validation prc-auc = 0.346472
Epoch 94
Loss = 6.0640e-02, PNorm = 84.7099, GNorm = 1.4590, lr_0 = 8.6198e-04
Validation prc-auc = 0.360081
Epoch 95
Validation prc-auc = 0.356844
Epoch 96
Validation prc-auc = 0.342179
Epoch 97
Validation prc-auc = 0.285905
Epoch 98
Validation prc-auc = 0.196533
Epoch 99
Loss = 5.2903e-02, PNorm = 84.9901, GNorm = 0.4358, lr_0 = 8.5461e-04
Validation prc-auc = 0.229233
Epoch 100
Validation prc-auc = 0.249626
Epoch 101
Validation prc-auc = 0.258652
Epoch 102
Validation prc-auc = 0.271943
Epoch 103
Validation prc-auc = 0.290447
Epoch 104
Loss = 2.2823e-02, PNorm = 85.2376, GNorm = 0.5505, lr_0 = 8.4730e-04
Validation prc-auc = 0.321223
Epoch 105
Validation prc-auc = 0.308526
Epoch 106
Validation prc-auc = 0.312397
Epoch 107
Validation prc-auc = 0.292327
Epoch 108
Validation prc-auc = 0.325420
Epoch 109
Loss = 2.6738e-02, PNorm = 85.4408, GNorm = 0.6383, lr_0 = 8.4006e-04
Validation prc-auc = 0.325156
Epoch 110
Validation prc-auc = 0.317045
Epoch 111
Validation prc-auc = 0.318840
Epoch 112
Validation prc-auc = 0.311052
Epoch 113
Validation prc-auc = 0.269121
Epoch 114
Loss = 1.9127e-02, PNorm = 85.6330, GNorm = 1.2177, lr_0 = 8.3288e-04
Validation prc-auc = 0.266165
Epoch 115
Validation prc-auc = 0.249533
Epoch 116
Validation prc-auc = 0.220490
Epoch 117
Validation prc-auc = 0.217333
Epoch 118
Validation prc-auc = 0.242705
Epoch 119
Loss = 2.5329e-03, PNorm = 85.7843, GNorm = 0.0279, lr_0 = 8.2576e-04
Validation prc-auc = 0.251771
Epoch 120
Validation prc-auc = 0.254955
Epoch 121
Validation prc-auc = 0.253266
Epoch 122
Validation prc-auc = 0.236312
Epoch 123
Validation prc-auc = 0.235802
Epoch 124
Loss = 4.2049e-03, PNorm = 85.9025, GNorm = 0.0175, lr_0 = 8.1870e-04
Validation prc-auc = 0.245836
Epoch 125
Validation prc-auc = 0.262948
Epoch 126
Validation prc-auc = 0.274168
Epoch 127
Validation prc-auc = 0.286154
Epoch 128
Validation prc-auc = 0.286708
Epoch 129
Loss = 6.6386e-03, PNorm = 86.0234, GNorm = 0.1598, lr_0 = 8.1170e-04
Validation prc-auc = 0.286670
Epoch 130
Validation prc-auc = 0.280666
Epoch 131
Validation prc-auc = 0.276780
Epoch 132
Validation prc-auc = 0.271997
Epoch 133
Validation prc-auc = 0.271172
Epoch 134
Loss = 2.2648e-03, PNorm = 86.1266, GNorm = 0.0519, lr_0 = 8.0476e-04
Validation prc-auc = 0.273403
Epoch 135
Validation prc-auc = 0.284740
Epoch 136
Validation prc-auc = 0.281803
Epoch 137
Validation prc-auc = 0.269278
Epoch 138
Validation prc-auc = 0.268204
Epoch 139
Loss = 5.9064e-03, PNorm = 86.1971, GNorm = 0.1472, lr_0 = 7.9788e-04
Validation prc-auc = 0.248496
Epoch 140
Validation prc-auc = 0.224663
Epoch 141
Validation prc-auc = 0.237987
Epoch 142
Validation prc-auc = 0.251706
Epoch 143
Validation prc-auc = 0.263135
Epoch 144
Loss = 5.0731e-03, PNorm = 86.2741, GNorm = 0.1962, lr_0 = 7.9106e-04
Validation prc-auc = 0.269497
Epoch 145
Validation prc-auc = 0.288364
Epoch 146
Validation prc-auc = 0.299853
Epoch 147
Validation prc-auc = 0.300752
Epoch 148
Validation prc-auc = 0.307451
Epoch 149
Loss = 4.7443e-03, PNorm = 86.3435, GNorm = 0.1533, lr_0 = 7.8430e-04
Validation prc-auc = 0.308677
Epoch 150
Validation prc-auc = 0.307553
Epoch 151
Validation prc-auc = 0.303300
Epoch 152
Validation prc-auc = 0.301494
Epoch 153
Validation prc-auc = 0.290292
Epoch 154
Loss = 4.1294e-04, PNorm = 86.4086, GNorm = 0.0088, lr_0 = 7.7759e-04
Validation prc-auc = 0.287722
Epoch 155
Validation prc-auc = 0.276216
Epoch 156
Validation prc-auc = 0.275685
Epoch 157
Validation prc-auc = 0.275287
Epoch 158
Validation prc-auc = 0.274977
Epoch 159
Loss = 5.0478e-04, PNorm = 86.4433, GNorm = 0.0198, lr_0 = 7.7095e-04
Validation prc-auc = 0.285617
Epoch 160
Validation prc-auc = 0.291283
Epoch 161
Validation prc-auc = 0.294036
Epoch 162
Validation prc-auc = 0.295108
Epoch 163
Validation prc-auc = 0.294899
Epoch 164
Loss = 1.1425e-03, PNorm = 86.4712, GNorm = 0.0720, lr_0 = 7.6436e-04
Validation prc-auc = 0.303949
Epoch 165
Validation prc-auc = 0.304318
Epoch 166
Validation prc-auc = 0.305851
Epoch 167
Validation prc-auc = 0.308465
Epoch 168
Validation prc-auc = 0.309083
Epoch 169
Loss = 2.4120e-03, PNorm = 86.5002, GNorm = 0.1663, lr_0 = 7.5782e-04
Validation prc-auc = 0.304228
Epoch 170
Validation prc-auc = 0.302654
Epoch 171
Validation prc-auc = 0.300803
Epoch 172
Validation prc-auc = 0.288076
Epoch 173
Validation prc-auc = 0.266056
Epoch 174
Loss = 5.1762e-03, PNorm = 86.5392, GNorm = 0.3394, lr_0 = 7.5134e-04
Validation prc-auc = 0.253052
Epoch 175
Validation prc-auc = 0.249086
Epoch 176
Validation prc-auc = 0.240232
Epoch 177
Validation prc-auc = 0.236760
Epoch 178
Validation prc-auc = 0.245920
Epoch 179
Loss = 4.4986e-04, PNorm = 86.6111, GNorm = 0.0509, lr_0 = 7.4492e-04
Validation prc-auc = 0.255787
Epoch 180
Validation prc-auc = 0.253410
Epoch 181
Validation prc-auc = 0.247341
Epoch 182
Validation prc-auc = 0.251789
Epoch 183
Validation prc-auc = 0.252916
Epoch 184
Loss = 3.4637e-04, PNorm = 86.6513, GNorm = 0.0046, lr_0 = 7.3855e-04
Validation prc-auc = 0.252702
Epoch 185
Validation prc-auc = 0.255836
Epoch 186
Validation prc-auc = 0.261791
Epoch 187
Validation prc-auc = 0.261721
Epoch 188
Validation prc-auc = 0.261903
Epoch 189
Loss = 4.1657e-04, PNorm = 86.6766, GNorm = 0.0157, lr_0 = 7.3224e-04
Validation prc-auc = 0.261880
Epoch 190
Validation prc-auc = 0.261832
Epoch 191
Validation prc-auc = 0.261858
Epoch 192
Validation prc-auc = 0.263395
Epoch 193
Validation prc-auc = 0.263395
Epoch 194
Loss = 3.9351e-04, PNorm = 86.6943, GNorm = 0.0241, lr_0 = 7.2598e-04
Validation prc-auc = 0.263539
Epoch 195
Validation prc-auc = 0.263459
Epoch 196
Validation prc-auc = 0.257308
Epoch 197
Validation prc-auc = 0.257389
Epoch 198
Validation prc-auc = 0.257389
Epoch 199
Loss = 1.3199e-04, PNorm = 86.7039, GNorm = 0.0050, lr_0 = 7.1977e-04
Validation prc-auc = 0.255762
Epoch 200
Validation prc-auc = 0.255898
Epoch 201
Validation prc-auc = 0.255898
Epoch 202
Validation prc-auc = 0.256025
Epoch 203
Validation prc-auc = 0.256097
Epoch 204
Loss = 1.1349e-04, PNorm = 86.7099, GNorm = 0.0032, lr_0 = 7.1362e-04
Validation prc-auc = 0.256097
Epoch 205
Validation prc-auc = 0.256099
Epoch 206
Validation prc-auc = 0.256112
Epoch 207
Validation prc-auc = 0.256198
Epoch 208
Validation prc-auc = 0.249690
Epoch 209
Loss = 8.2639e-05, PNorm = 86.7144, GNorm = 0.0027, lr_0 = 7.0752e-04
Validation prc-auc = 0.249690
Epoch 210
Validation prc-auc = 0.249690
Epoch 211
Validation prc-auc = 0.249690
Epoch 212
Validation prc-auc = 0.249690
Epoch 213
Validation prc-auc = 0.249779
Epoch 214
Loss = 8.4569e-05, PNorm = 86.7185, GNorm = 0.0030, lr_0 = 7.0147e-04
Validation prc-auc = 0.249779
Epoch 215
Validation prc-auc = 0.249762
Epoch 216
Validation prc-auc = 0.249890
Epoch 217
Validation prc-auc = 0.249953
Epoch 218
Validation prc-auc = 0.249953
Epoch 219
Loss = 8.3627e-05, PNorm = 86.7223, GNorm = 0.0024, lr_0 = 6.9548e-04
Validation prc-auc = 0.256533
Epoch 220
Validation prc-auc = 0.256463
Epoch 221
Validation prc-auc = 0.256555
Epoch 222
Validation prc-auc = 0.256555
Epoch 223
Validation prc-auc = 0.257064
Epoch 224
Loss = 7.8145e-05, PNorm = 86.7260, GNorm = 0.0025, lr_0 = 6.8953e-04
Validation prc-auc = 0.257064
Epoch 225
Validation prc-auc = 0.257129
Epoch 226
Validation prc-auc = 0.257129
Epoch 227
Validation prc-auc = 0.257129
Epoch 228
Validation prc-auc = 0.257129
Epoch 229
Loss = 1.1600e-04, PNorm = 86.7296, GNorm = 0.0026, lr_0 = 6.8364e-04
Validation prc-auc = 0.257129
Epoch 230
Validation prc-auc = 0.257129
Epoch 231
Validation prc-auc = 0.257129
Epoch 232
Validation prc-auc = 0.257129
Epoch 233
Validation prc-auc = 0.257129
Epoch 234
Loss = 1.8573e-04, PNorm = 86.7332, GNorm = 0.0031, lr_0 = 6.7779e-04
Validation prc-auc = 0.257065
Epoch 235
Validation prc-auc = 0.257065
Epoch 236
Validation prc-auc = 0.257199
Epoch 237
Validation prc-auc = 0.257277
Epoch 238
Validation prc-auc = 0.257344
Epoch 239
Loss = 9.2596e-05, PNorm = 86.7372, GNorm = 0.0019, lr_0 = 6.7200e-04
Validation prc-auc = 0.257278
Epoch 240
Validation prc-auc = 0.257255
Epoch 241
Validation prc-auc = 0.257255
Epoch 242
Validation prc-auc = 0.257255
Epoch 243
Validation prc-auc = 0.257255
Epoch 244
Loss = 6.1965e-05, PNorm = 86.7413, GNorm = 0.0012, lr_0 = 6.6625e-04
Validation prc-auc = 0.257184
Epoch 245
Validation prc-auc = 0.257184
Epoch 246
Validation prc-auc = 0.257115
Epoch 247
Validation prc-auc = 0.257206
Epoch 248
Validation prc-auc = 0.257206
Epoch 249
Loss = 6.6423e-05, PNorm = 86.7452, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.257139
Epoch 250
Validation prc-auc = 0.257139
Epoch 251
Validation prc-auc = 0.257097
Epoch 252
Validation prc-auc = 0.257097
Epoch 253
Validation prc-auc = 0.257097
Epoch 254
Loss = 4.9712e-05, PNorm = 86.7488, GNorm = 0.0019, lr_0 = 6.5491e-04
Validation prc-auc = 0.257082
Epoch 255
Validation prc-auc = 0.257006
Epoch 256
Validation prc-auc = 0.257082
Epoch 257
Validation prc-auc = 0.257071
Epoch 258
Validation prc-auc = 0.257147
Epoch 259
Loss = 5.1863e-05, PNorm = 86.7524, GNorm = 0.0013, lr_0 = 6.4931e-04
Validation prc-auc = 0.256987
Epoch 260
Validation prc-auc = 0.256963
Epoch 261
Validation prc-auc = 0.256963
Epoch 262
Validation prc-auc = 0.256963
Epoch 263
Validation prc-auc = 0.257030
Epoch 264
Loss = 1.9087e-04, PNorm = 86.7558, GNorm = 0.0013, lr_0 = 6.4376e-04
Validation prc-auc = 0.257163
Epoch 265
Validation prc-auc = 0.257311
Epoch 266
Validation prc-auc = 0.257779
Epoch 267
Validation prc-auc = 0.257779
Epoch 268
Validation prc-auc = 0.257846
Epoch 269
Loss = 7.9699e-05, PNorm = 86.7595, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.263997
Epoch 270
Validation prc-auc = 0.263997
Epoch 271
Validation prc-auc = 0.264088
Epoch 272
Validation prc-auc = 0.264019
Epoch 273
Validation prc-auc = 0.264019
Epoch 274
Loss = 4.5145e-05, PNorm = 86.7629, GNorm = 0.0011, lr_0 = 6.3280e-04
Validation prc-auc = 0.264019
Epoch 275
Validation prc-auc = 0.257869
Epoch 276
Validation prc-auc = 0.264510
Epoch 277
Validation prc-auc = 0.264510
Epoch 278
Validation prc-auc = 0.264443
Epoch 279
Loss = 3.9168e-05, PNorm = 86.7663, GNorm = 0.0010, lr_0 = 6.2739e-04
Validation prc-auc = 0.264443
Epoch 280
Validation prc-auc = 0.264443
Epoch 281
Validation prc-auc = 0.264443
Epoch 282
Validation prc-auc = 0.264537
Epoch 283
Validation prc-auc = 0.264537
Epoch 284
Loss = 1.1023e-04, PNorm = 86.7693, GNorm = 0.0100, lr_0 = 6.2203e-04
Validation prc-auc = 0.264537
Epoch 285
Validation prc-auc = 0.264523
Epoch 286
Validation prc-auc = 0.264590
Epoch 287
Validation prc-auc = 0.264659
Epoch 288
Validation prc-auc = 0.264728
Epoch 289
Loss = 3.2019e-05, PNorm = 86.7725, GNorm = 0.0013, lr_0 = 6.1671e-04
Validation prc-auc = 0.264728
Epoch 290
Validation prc-auc = 0.264728
Epoch 291
Validation prc-auc = 0.264728
Epoch 292
Validation prc-auc = 0.264728
Epoch 293
Validation prc-auc = 0.264718
Epoch 294
Loss = 3.1254e-05, PNorm = 86.7757, GNorm = 0.0010, lr_0 = 6.1144e-04
Validation prc-auc = 0.264718
Epoch 295
Validation prc-auc = 0.264718
Epoch 296
Validation prc-auc = 0.264609
Epoch 297
Validation prc-auc = 0.264609
Epoch 298
Validation prc-auc = 0.264978
Epoch 299
Loss = 3.0260e-05, PNorm = 86.7789, GNorm = 0.0009, lr_0 = 6.0621e-04
Validation prc-auc = 0.264978
Model 0 best validation prc-auc = 0.490877 on epoch 8
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.387804
Ensemble test prc-auc = 0.387804
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.421469
Epoch 1
Validation prc-auc = 0.423979
Epoch 2
Validation prc-auc = 0.439751
Epoch 3
Validation prc-auc = 0.465896
Epoch 4
Loss = 7.3687e-01, PNorm = 78.5527, GNorm = 8.0153, lr_0 = 6.5000e-04
Validation prc-auc = 0.443333
Epoch 5
Validation prc-auc = 0.463636
Epoch 6
Validation prc-auc = 0.433615
Epoch 7
Validation prc-auc = 0.319327
Epoch 8
Validation prc-auc = 0.433522
Epoch 9
Loss = 5.4549e-01, PNorm = 78.7983, GNorm = 1.2378, lr_0 = 9.9743e-04
Validation prc-auc = 0.439056
Epoch 10
Validation prc-auc = 0.451785
Epoch 11
Validation prc-auc = 0.362315
Epoch 12
Validation prc-auc = 0.331306
Epoch 13
Validation prc-auc = 0.316753
Epoch 14
Loss = 5.2377e-01, PNorm = 79.1024, GNorm = 0.8953, lr_0 = 9.8890e-04
Validation prc-auc = 0.320442
Epoch 15
Validation prc-auc = 0.295246
Epoch 16
Validation prc-auc = 0.325186
Epoch 17
Validation prc-auc = 0.353658
Epoch 18
Validation prc-auc = 0.353847
Epoch 19
Loss = 4.4552e-01, PNorm = 79.4384, GNorm = 1.8933, lr_0 = 9.8045e-04
Validation prc-auc = 0.304821
Epoch 20
Validation prc-auc = 0.304510
Epoch 21
Validation prc-auc = 0.334042
Epoch 22
Validation prc-auc = 0.388249
Epoch 23
Validation prc-auc = 0.385059
Epoch 24
Loss = 3.6079e-01, PNorm = 79.7877, GNorm = 2.4063, lr_0 = 9.7207e-04
Validation prc-auc = 0.456953
Epoch 25
Validation prc-auc = 0.445516
Epoch 26
Validation prc-auc = 0.298666
Epoch 27
Validation prc-auc = 0.266809
Epoch 28
Validation prc-auc = 0.268973
Epoch 29
Loss = 3.4658e-01, PNorm = 80.1289, GNorm = 0.8745, lr_0 = 9.6376e-04
Validation prc-auc = 0.318488
Epoch 30
Validation prc-auc = 0.326917
Epoch 31
Validation prc-auc = 0.329532
Epoch 32
Validation prc-auc = 0.325767
Epoch 33
Validation prc-auc = 0.166782
Epoch 34
Loss = 2.9142e-01, PNorm = 80.4997, GNorm = 1.4784, lr_0 = 9.5552e-04
Validation prc-auc = 0.263719
Epoch 35
Validation prc-auc = 0.281547
Epoch 36
Validation prc-auc = 0.323984
Epoch 37
Validation prc-auc = 0.341547
Epoch 38
Validation prc-auc = 0.352572
Epoch 39
Loss = 3.0176e-01, PNorm = 80.9185, GNorm = 1.8169, lr_0 = 9.4735e-04
Validation prc-auc = 0.345094
Epoch 40
Validation prc-auc = 0.307132
Epoch 41
Validation prc-auc = 0.325801
Epoch 42
Validation prc-auc = 0.341158
Epoch 43
Validation prc-auc = 0.332618
Epoch 44
Loss = 1.5395e-01, PNorm = 81.3193, GNorm = 1.4727, lr_0 = 9.3925e-04
Validation prc-auc = 0.346512
Epoch 45
Validation prc-auc = 0.314487
Epoch 46
Validation prc-auc = 0.321420
Epoch 47
Validation prc-auc = 0.331676
Epoch 48
Validation prc-auc = 0.325380
Epoch 49
Loss = 8.4553e-02, PNorm = 81.7248, GNorm = 0.4701, lr_0 = 9.3122e-04
Validation prc-auc = 0.336953
Epoch 50
Validation prc-auc = 0.317782
Epoch 51
Validation prc-auc = 0.334362
Epoch 52
Validation prc-auc = 0.344895
Epoch 53
Validation prc-auc = 0.338960
Epoch 54
Loss = 1.1898e-01, PNorm = 82.1275, GNorm = 1.7424, lr_0 = 9.2326e-04
Validation prc-auc = 0.312097
Epoch 55
Validation prc-auc = 0.224829
Epoch 56
Validation prc-auc = 0.211702
Epoch 57
Validation prc-auc = 0.228625
Epoch 58
Validation prc-auc = 0.249902
Epoch 59
Loss = 1.3664e-01, PNorm = 82.5256, GNorm = 0.8003, lr_0 = 9.1537e-04
Validation prc-auc = 0.315017
Epoch 60
Validation prc-auc = 0.171745
Epoch 61
Validation prc-auc = 0.254731
Epoch 62
Validation prc-auc = 0.270841
Epoch 63
Validation prc-auc = 0.216812
Epoch 64
Loss = 2.4207e-01, PNorm = 82.9790, GNorm = 3.0624, lr_0 = 9.0754e-04
Validation prc-auc = 0.303712
Epoch 65
Validation prc-auc = 0.300287
Epoch 66
Validation prc-auc = 0.227536
Epoch 67
Validation prc-auc = 0.234641
Epoch 68
Validation prc-auc = 0.293496
Epoch 69
Loss = 4.8540e-02, PNorm = 83.3779, GNorm = 0.2424, lr_0 = 8.9978e-04
Validation prc-auc = 0.277111
Epoch 70
Validation prc-auc = 0.254592
Epoch 71
Validation prc-auc = 0.258545
Epoch 72
Validation prc-auc = 0.237237
Epoch 73
Validation prc-auc = 0.219126
Epoch 74
Loss = 1.3329e-01, PNorm = 83.6973, GNorm = 2.4291, lr_0 = 8.9209e-04
Validation prc-auc = 0.237532
Epoch 75
Validation prc-auc = 0.266272
Epoch 76
Validation prc-auc = 0.282868
Epoch 77
Validation prc-auc = 0.245745
Epoch 78
Validation prc-auc = 0.281975
Epoch 79
Loss = 3.0196e-02, PNorm = 83.9751, GNorm = 0.2708, lr_0 = 8.8447e-04
Validation prc-auc = 0.328357
Epoch 80
Validation prc-auc = 0.321928
Epoch 81
Validation prc-auc = 0.235448
Epoch 82
Validation prc-auc = 0.195459
Epoch 83
Validation prc-auc = 0.192653
Epoch 84
Loss = 2.9629e-02, PNorm = 84.2209, GNorm = 0.1373, lr_0 = 8.7691e-04
Validation prc-auc = 0.239309
Epoch 85
Validation prc-auc = 0.304257
Epoch 86
Validation prc-auc = 0.333307
Epoch 87
Validation prc-auc = 0.289372
Epoch 88
Validation prc-auc = 0.236526
Epoch 89
Loss = 2.6510e-02, PNorm = 84.4304, GNorm = 0.0461, lr_0 = 8.6941e-04
Validation prc-auc = 0.198907
Epoch 90
Validation prc-auc = 0.251381
Epoch 91
Validation prc-auc = 0.278850
Epoch 92
Validation prc-auc = 0.279741
Epoch 93
Validation prc-auc = 0.248908
Epoch 94
Loss = 4.4098e-02, PNorm = 84.6117, GNorm = 0.1094, lr_0 = 8.6198e-04
Validation prc-auc = 0.217561
Epoch 95
Validation prc-auc = 0.220587
Epoch 96
Validation prc-auc = 0.230401
Epoch 97
Validation prc-auc = 0.225088
Epoch 98
Validation prc-auc = 0.193584
Epoch 99
Loss = 1.4977e-02, PNorm = 84.7991, GNorm = 0.3501, lr_0 = 8.5461e-04
Validation prc-auc = 0.176868
Epoch 100
Validation prc-auc = 0.170880
Epoch 101
Validation prc-auc = 0.172012
Epoch 102
Validation prc-auc = 0.185316
Epoch 103
Validation prc-auc = 0.199213
Epoch 104
Loss = 5.0745e-03, PNorm = 84.9753, GNorm = 0.2658, lr_0 = 8.4730e-04
Validation prc-auc = 0.211431
Epoch 105
Validation prc-auc = 0.217693
Epoch 106
Validation prc-auc = 0.230831
Epoch 107
Validation prc-auc = 0.240978
Epoch 108
Validation prc-auc = 0.263327
Epoch 109
Loss = 1.2869e-02, PNorm = 85.1049, GNorm = 1.0465, lr_0 = 8.4006e-04
Validation prc-auc = 0.267876
Epoch 110
Validation prc-auc = 0.234446
Epoch 111
Validation prc-auc = 0.221977
Epoch 112
Validation prc-auc = 0.209011
Epoch 113
Validation prc-auc = 0.207965
Epoch 114
Loss = 2.2331e-03, PNorm = 85.1923, GNorm = 0.1035, lr_0 = 8.3288e-04
Validation prc-auc = 0.216497
Epoch 115
Validation prc-auc = 0.226325
Epoch 116
Validation prc-auc = 0.227724
Epoch 117
Validation prc-auc = 0.235186
Epoch 118
Validation prc-auc = 0.237335
Epoch 119
Loss = 2.6992e-02, PNorm = 85.2644, GNorm = 0.3377, lr_0 = 8.2576e-04
Validation prc-auc = 0.266233
Epoch 120
Validation prc-auc = 0.261343
Epoch 121
Validation prc-auc = 0.233877
Epoch 122
Validation prc-auc = 0.229014
Epoch 123
Validation prc-auc = 0.238096
Epoch 124
Loss = 6.5969e-02, PNorm = 85.3855, GNorm = 1.1910, lr_0 = 8.1870e-04
Validation prc-auc = 0.269577
Epoch 125
Validation prc-auc = 0.276810
Epoch 126
Validation prc-auc = 0.269922
Epoch 127
Validation prc-auc = 0.243705
Epoch 128
Validation prc-auc = 0.224063
Epoch 129
Loss = 1.1894e-02, PNorm = 85.5518, GNorm = 0.3324, lr_0 = 8.1170e-04
Validation prc-auc = 0.202577
Epoch 130
Validation prc-auc = 0.213873
Epoch 131
Validation prc-auc = 0.218412
Epoch 132
Validation prc-auc = 0.245407
Epoch 133
Validation prc-auc = 0.270409
Epoch 134
Loss = 1.1480e-02, PNorm = 85.7105, GNorm = 0.5125, lr_0 = 8.0476e-04
Validation prc-auc = 0.270824
Epoch 135
Validation prc-auc = 0.270837
Epoch 136
Validation prc-auc = 0.268830
Epoch 137
Validation prc-auc = 0.260741
Epoch 138
Validation prc-auc = 0.263618
Epoch 139
Loss = 1.1048e-02, PNorm = 85.8041, GNorm = 0.5878, lr_0 = 7.9788e-04
Validation prc-auc = 0.224960
Epoch 140
Validation prc-auc = 0.222092
Epoch 141
Validation prc-auc = 0.220771
Epoch 142
Validation prc-auc = 0.220701
Epoch 143
Validation prc-auc = 0.214179
Epoch 144
Loss = 1.4845e-03, PNorm = 85.9090, GNorm = 0.1000, lr_0 = 7.9106e-04
Validation prc-auc = 0.215867
Epoch 145
Validation prc-auc = 0.214265
Epoch 146
Validation prc-auc = 0.216375
Epoch 147
Validation prc-auc = 0.213805
Epoch 148
Validation prc-auc = 0.214617
Epoch 149
Loss = 1.1243e-03, PNorm = 85.9787, GNorm = 0.0499, lr_0 = 7.8430e-04
Validation prc-auc = 0.212677
Epoch 150
Validation prc-auc = 0.217421
Epoch 151
Validation prc-auc = 0.225611
Epoch 152
Validation prc-auc = 0.226140
Epoch 153
Validation prc-auc = 0.226447
Epoch 154
Loss = 6.7331e-04, PNorm = 86.0258, GNorm = 0.0165, lr_0 = 7.7759e-04
Validation prc-auc = 0.225323
Epoch 155
Validation prc-auc = 0.228061
Epoch 156
Validation prc-auc = 0.228166
Epoch 157
Validation prc-auc = 0.231046
Epoch 158
Validation prc-auc = 0.230086
Epoch 159
Loss = 5.1876e-04, PNorm = 86.0613, GNorm = 0.0073, lr_0 = 7.7095e-04
Validation prc-auc = 0.230086
Epoch 160
Validation prc-auc = 0.231706
Epoch 161
Validation prc-auc = 0.230949
Epoch 162
Validation prc-auc = 0.226674
Epoch 163
Validation prc-auc = 0.224061
Epoch 164
Loss = 5.7099e-04, PNorm = 86.0859, GNorm = 0.0521, lr_0 = 7.6436e-04
Validation prc-auc = 0.229333
Epoch 165
Validation prc-auc = 0.235687
Epoch 166
Validation prc-auc = 0.235814
Epoch 167
Validation prc-auc = 0.236074
Epoch 168
Validation prc-auc = 0.235992
Epoch 169
Loss = 5.0852e-04, PNorm = 86.1070, GNorm = 0.0154, lr_0 = 7.5782e-04
Validation prc-auc = 0.235922
Epoch 170
Validation prc-auc = 0.232089
Epoch 171
Validation prc-auc = 0.231757
Epoch 172
Validation prc-auc = 0.231473
Epoch 173
Validation prc-auc = 0.231533
Epoch 174
Loss = 5.2526e-04, PNorm = 86.1244, GNorm = 0.0538, lr_0 = 7.5134e-04
Validation prc-auc = 0.231316
Epoch 175
Validation prc-auc = 0.231248
Epoch 176
Validation prc-auc = 0.231120
Epoch 177
Validation prc-auc = 0.230419
Epoch 178
Validation prc-auc = 0.230486
Epoch 179
Loss = 2.8719e-04, PNorm = 86.1390, GNorm = 0.0099, lr_0 = 7.4492e-04
Validation prc-auc = 0.230486
Epoch 180
Validation prc-auc = 0.230486
Epoch 181
Validation prc-auc = 0.230697
Epoch 182
Validation prc-auc = 0.230764
Epoch 183
Validation prc-auc = 0.230314
Epoch 184
Loss = 1.7340e-04, PNorm = 86.1509, GNorm = 0.0039, lr_0 = 7.3855e-04
Validation prc-auc = 0.230314
Epoch 185
Validation prc-auc = 0.234203
Epoch 186
Validation prc-auc = 0.234203
Epoch 187
Validation prc-auc = 0.234422
Epoch 188
Validation prc-auc = 0.234355
Epoch 189
Loss = 1.6721e-04, PNorm = 86.1599, GNorm = 0.0051, lr_0 = 7.3224e-04
Validation prc-auc = 0.234355
Epoch 190
Validation prc-auc = 0.234288
Epoch 191
Validation prc-auc = 0.238891
Epoch 192
Validation prc-auc = 0.238958
Epoch 193
Validation prc-auc = 0.238749
Epoch 194
Loss = 2.0894e-04, PNorm = 86.1678, GNorm = 0.0071, lr_0 = 7.2598e-04
Validation prc-auc = 0.238749
Epoch 195
Validation prc-auc = 0.238815
Epoch 196
Validation prc-auc = 0.238585
Epoch 197
Validation prc-auc = 0.238585
Epoch 198
Validation prc-auc = 0.238706
Epoch 199
Loss = 7.1296e-05, PNorm = 86.1752, GNorm = 0.0019, lr_0 = 7.1977e-04
Validation prc-auc = 0.240320
Epoch 200
Validation prc-auc = 0.240320
Epoch 201
Validation prc-auc = 0.240320
Epoch 202
Validation prc-auc = 0.240320
Epoch 203
Validation prc-auc = 0.240179
Epoch 204
Loss = 2.1907e-04, PNorm = 86.1820, GNorm = 0.0056, lr_0 = 7.1362e-04
Validation prc-auc = 0.239934
Epoch 205
Validation prc-auc = 0.240002
Epoch 206
Validation prc-auc = 0.239845
Epoch 207
Validation prc-auc = 0.239845
Epoch 208
Validation prc-auc = 0.239710
Epoch 209
Loss = 5.3842e-05, PNorm = 86.1887, GNorm = 0.0024, lr_0 = 7.0752e-04
Validation prc-auc = 0.239772
Epoch 210
Validation prc-auc = 0.239772
Epoch 211
Validation prc-auc = 0.239898
Epoch 212
Validation prc-auc = 0.239898
Epoch 213
Validation prc-auc = 0.239965
Epoch 214
Loss = 1.1752e-04, PNorm = 86.1954, GNorm = 0.0012, lr_0 = 7.0147e-04
Validation prc-auc = 0.240030
Epoch 215
Validation prc-auc = 0.240170
Epoch 216
Validation prc-auc = 0.240849
Epoch 217
Validation prc-auc = 0.240921
Epoch 218
Validation prc-auc = 0.241056
Epoch 219
Loss = 8.7771e-05, PNorm = 86.2030, GNorm = 0.0032, lr_0 = 6.9548e-04
Validation prc-auc = 0.244760
Epoch 220
Validation prc-auc = 0.244760
Epoch 221
Validation prc-auc = 0.244690
Epoch 222
Validation prc-auc = 0.244626
Epoch 223
Validation prc-auc = 0.244837
Epoch 224
Loss = 7.8586e-05, PNorm = 86.2105, GNorm = 0.0031, lr_0 = 6.8953e-04
Validation prc-auc = 0.244524
Epoch 225
Validation prc-auc = 0.244362
Epoch 226
Validation prc-auc = 0.249124
Epoch 227
Validation prc-auc = 0.249188
Epoch 228
Validation prc-auc = 0.249326
Epoch 229
Loss = 5.3647e-05, PNorm = 86.2180, GNorm = 0.0025, lr_0 = 6.8364e-04
Validation prc-auc = 0.249394
Epoch 230
Validation prc-auc = 0.249394
Epoch 231
Validation prc-auc = 0.249323
Epoch 232
Validation prc-auc = 0.249323
Epoch 233
Validation prc-auc = 0.249774
Epoch 234
Loss = 4.6062e-05, PNorm = 86.2253, GNorm = 0.0011, lr_0 = 6.7779e-04
Validation prc-auc = 0.249422
Epoch 235
Validation prc-auc = 0.249270
Epoch 236
Validation prc-auc = 0.249270
Epoch 237
Validation prc-auc = 0.249123
Epoch 238
Validation prc-auc = 0.249323
Epoch 239
Loss = 4.2064e-05, PNorm = 86.2315, GNorm = 0.0017, lr_0 = 6.7200e-04
Validation prc-auc = 0.249323
Epoch 240
Validation prc-auc = 0.249186
Epoch 241
Validation prc-auc = 0.248821
Epoch 242
Validation prc-auc = 0.249550
Epoch 243
Validation prc-auc = 0.249271
Epoch 244
Loss = 5.0311e-05, PNorm = 86.2382, GNorm = 0.0018, lr_0 = 6.6625e-04
Validation prc-auc = 0.249340
Epoch 245
Validation prc-auc = 0.249273
Epoch 246
Validation prc-auc = 0.249273
Epoch 247
Validation prc-auc = 0.249343
Epoch 248
Validation prc-auc = 0.249351
Epoch 249
Loss = 2.3754e-05, PNorm = 86.2447, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.249420
Epoch 250
Validation prc-auc = 0.249700
Epoch 251
Validation prc-auc = 0.250617
Epoch 252
Validation prc-auc = 0.250836
Epoch 253
Validation prc-auc = 0.254120
Epoch 254
Loss = 1.9915e-05, PNorm = 86.2511, GNorm = 0.0009, lr_0 = 6.5491e-04
Validation prc-auc = 0.253981
Epoch 255
Validation prc-auc = 0.252300
Epoch 256
Validation prc-auc = 0.252157
Epoch 257
Validation prc-auc = 0.251569
Epoch 258
Validation prc-auc = 0.251691
Epoch 259
Loss = 1.3458e-04, PNorm = 86.2563, GNorm = 0.0071, lr_0 = 6.4931e-04
Validation prc-auc = 0.254110
Epoch 260
Validation prc-auc = 0.254779
Epoch 261
Validation prc-auc = 0.258094
Epoch 262
Validation prc-auc = 0.258455
Epoch 263
Validation prc-auc = 0.258931
Epoch 264
Loss = 2.1682e-05, PNorm = 86.2614, GNorm = 0.0007, lr_0 = 6.4376e-04
Validation prc-auc = 0.258857
Epoch 265
Validation prc-auc = 0.258299
Epoch 266
Validation prc-auc = 0.257925
Epoch 267
Validation prc-auc = 0.251979
Epoch 268
Validation prc-auc = 0.252051
Epoch 269
Loss = 1.4931e-05, PNorm = 86.2663, GNorm = 0.0010, lr_0 = 6.3826e-04
Validation prc-auc = 0.251905
Epoch 270
Validation prc-auc = 0.251832
Epoch 271
Validation prc-auc = 0.251908
Epoch 272
Validation prc-auc = 0.254670
Epoch 273
Validation prc-auc = 0.254670
Epoch 274
Loss = 1.6231e-05, PNorm = 86.2704, GNorm = 0.0012, lr_0 = 6.3280e-04
Validation prc-auc = 0.254670
Epoch 275
Validation prc-auc = 0.254597
Epoch 276
Validation prc-auc = 0.254526
Epoch 277
Validation prc-auc = 0.258474
Epoch 278
Validation prc-auc = 0.262857
Epoch 279
Loss = 1.1756e-05, PNorm = 86.2735, GNorm = 0.0004, lr_0 = 6.2739e-04
Validation prc-auc = 0.262866
Epoch 280
Validation prc-auc = 0.259768
Epoch 281
Validation prc-auc = 0.260306
Epoch 282
Validation prc-auc = 0.266757
Epoch 283
Validation prc-auc = 0.264661
Epoch 284
Loss = 1.2980e-05, PNorm = 86.2767, GNorm = 0.0006, lr_0 = 6.2203e-04
Validation prc-auc = 0.264400
Epoch 285
Validation prc-auc = 0.265007
Epoch 286
Validation prc-auc = 0.268412
Epoch 287
Validation prc-auc = 0.267190
Epoch 288
Validation prc-auc = 0.266879
Epoch 289
Loss = 7.5815e-06, PNorm = 86.2803, GNorm = 0.0003, lr_0 = 6.1671e-04
Validation prc-auc = 0.263334
Epoch 290
Validation prc-auc = 0.260106
Epoch 291
Validation prc-auc = 0.263176
Epoch 292
Validation prc-auc = 0.266582
Epoch 293
Validation prc-auc = 0.266857
Epoch 294
Loss = 2.3784e-05, PNorm = 86.2831, GNorm = 0.0008, lr_0 = 6.1144e-04
Validation prc-auc = 0.270101
Epoch 295
Validation prc-auc = 0.274080
Epoch 296
Validation prc-auc = 0.274169
Epoch 297
Validation prc-auc = 0.273711
Epoch 298
Validation prc-auc = 0.273629
Epoch 299
Loss = 1.2231e-05, PNorm = 86.2869, GNorm = 0.0006, lr_0 = 6.0621e-04
Validation prc-auc = 0.273753
Model 0 best validation prc-auc = 0.465896 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.381926
Ensemble test prc-auc = 0.381926
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.435708
Epoch 1
Validation prc-auc = 0.441723
Epoch 2
Validation prc-auc = 0.454004
Epoch 3
Validation prc-auc = 0.425858
Epoch 4
Loss = 8.8239e-01, PNorm = 78.5518, GNorm = 11.6588, lr_0 = 6.5000e-04
Validation prc-auc = 0.422802
Epoch 5
Validation prc-auc = 0.446360
Epoch 6
Validation prc-auc = 0.497502
Epoch 7
Validation prc-auc = 0.450476
Epoch 8
Validation prc-auc = 0.460662
Epoch 9
Loss = 5.7273e-01, PNorm = 78.8056, GNorm = 6.4887, lr_0 = 9.9743e-04
Validation prc-auc = 0.499893
Epoch 10
Validation prc-auc = 0.474553
Epoch 11
Validation prc-auc = 0.500709
Epoch 12
Validation prc-auc = 0.465800
Epoch 13
Validation prc-auc = 0.422143
Epoch 14
Loss = 4.7215e-01, PNorm = 79.0723, GNorm = 0.8683, lr_0 = 9.8890e-04
Validation prc-auc = 0.437083
Epoch 15
Validation prc-auc = 0.427480
Epoch 16
Validation prc-auc = 0.303268
Epoch 17
Validation prc-auc = 0.290890
Epoch 18
Validation prc-auc = 0.290655
Epoch 19
Loss = 4.8168e-01, PNorm = 79.3140, GNorm = 3.5564, lr_0 = 9.8045e-04
Validation prc-auc = 0.310672
Epoch 20
Validation prc-auc = 0.382304
Epoch 21
Validation prc-auc = 0.399580
Epoch 22
Validation prc-auc = 0.486670
Epoch 23
Validation prc-auc = 0.478546
Epoch 24
Loss = 3.9170e-01, PNorm = 79.5683, GNorm = 3.1018, lr_0 = 9.7207e-04
Validation prc-auc = 0.485464
Epoch 25
Validation prc-auc = 0.476910
Epoch 26
Validation prc-auc = 0.524092
Epoch 27
Validation prc-auc = 0.502251
Epoch 28
Validation prc-auc = 0.455351
Epoch 29
Loss = 4.1243e-01, PNorm = 79.8351, GNorm = 4.8494, lr_0 = 9.6376e-04
Validation prc-auc = 0.429718
Epoch 30
Validation prc-auc = 0.390266
Epoch 31
Validation prc-auc = 0.278632
Epoch 32
Validation prc-auc = 0.292380
Epoch 33
Validation prc-auc = 0.277029
Epoch 34
Loss = 3.6675e-01, PNorm = 80.1210, GNorm = 1.2056, lr_0 = 9.5552e-04
Validation prc-auc = 0.275304
Epoch 35
Validation prc-auc = 0.303900
Epoch 36
Validation prc-auc = 0.271541
Epoch 37
Validation prc-auc = 0.298444
Epoch 38
Validation prc-auc = 0.299185
Epoch 39
Loss = 2.2963e-01, PNorm = 80.4620, GNorm = 2.7669, lr_0 = 9.4735e-04
Validation prc-auc = 0.307856
Epoch 40
Validation prc-auc = 0.325169
Epoch 41
Validation prc-auc = 0.332641
Epoch 42
Validation prc-auc = 0.330515
Epoch 43
Validation prc-auc = 0.345459
Epoch 44
Loss = 2.7264e-01, PNorm = 80.7752, GNorm = 3.9265, lr_0 = 9.3925e-04
Validation prc-auc = 0.325186
Epoch 45
Validation prc-auc = 0.303068
Epoch 46
Validation prc-auc = 0.229226
Epoch 47
Validation prc-auc = 0.296760
Epoch 48
Validation prc-auc = 0.261483
Epoch 49
Loss = 1.7006e-01, PNorm = 81.1059, GNorm = 1.2697, lr_0 = 9.3122e-04
Validation prc-auc = 0.292475
Epoch 50
Validation prc-auc = 0.330260
Epoch 51
Validation prc-auc = 0.218820
Epoch 52
Validation prc-auc = 0.273329
Epoch 53
Validation prc-auc = 0.242914
Epoch 54
Loss = 1.9472e-01, PNorm = 81.4353, GNorm = 2.1499, lr_0 = 9.2326e-04
Validation prc-auc = 0.160746
Epoch 55
Validation prc-auc = 0.272838
Epoch 56
Validation prc-auc = 0.338849
Epoch 57
Validation prc-auc = 0.333858
Epoch 58
Validation prc-auc = 0.228223
Epoch 59
Loss = 1.6898e-01, PNorm = 81.7784, GNorm = 1.6197, lr_0 = 9.1537e-04
Validation prc-auc = 0.205948
Epoch 60
Validation prc-auc = 0.240472
Epoch 61
Validation prc-auc = 0.278798
Epoch 62
Validation prc-auc = 0.212128
Epoch 63
Validation prc-auc = 0.317435
Epoch 64
Loss = 8.3831e-02, PNorm = 82.1227, GNorm = 1.7327, lr_0 = 9.0754e-04
Validation prc-auc = 0.353449
Epoch 65
Validation prc-auc = 0.340085
Epoch 66
Validation prc-auc = 0.318553
Epoch 67
Validation prc-auc = 0.204701
Epoch 68
Validation prc-auc = 0.195564
Epoch 69
Loss = 9.3059e-02, PNorm = 82.4415, GNorm = 0.6657, lr_0 = 8.9978e-04
Validation prc-auc = 0.233861
Epoch 70
Validation prc-auc = 0.260961
Epoch 71
Validation prc-auc = 0.266063
Epoch 72
Validation prc-auc = 0.252819
Epoch 73
Validation prc-auc = 0.293240
Epoch 74
Loss = 3.4455e-02, PNorm = 82.7423, GNorm = 0.8372, lr_0 = 8.9209e-04
Validation prc-auc = 0.244053
Epoch 75
Validation prc-auc = 0.246577
Epoch 76
Validation prc-auc = 0.280393
Epoch 77
Validation prc-auc = 0.281936
Epoch 78
Validation prc-auc = 0.299902
Epoch 79
Loss = 6.6255e-02, PNorm = 83.0010, GNorm = 0.5624, lr_0 = 8.8447e-04
Validation prc-auc = 0.290115
Epoch 80
Validation prc-auc = 0.287320
Epoch 81
Validation prc-auc = 0.240154
Epoch 82
Validation prc-auc = 0.213205
Epoch 83
Validation prc-auc = 0.240780
Epoch 84
Loss = 5.5168e-02, PNorm = 83.2307, GNorm = 1.3664, lr_0 = 8.7691e-04
Validation prc-auc = 0.233837
Epoch 85
Validation prc-auc = 0.207928
Epoch 86
Validation prc-auc = 0.197484
Epoch 87
Validation prc-auc = 0.212902
Epoch 88
Validation prc-auc = 0.221800
Epoch 89
Loss = 2.5878e-02, PNorm = 83.4308, GNorm = 0.4328, lr_0 = 8.6941e-04
Validation prc-auc = 0.265031
Epoch 90
Validation prc-auc = 0.231669
Epoch 91
Validation prc-auc = 0.237319
Epoch 92
Validation prc-auc = 0.284283
Epoch 93
Validation prc-auc = 0.304206
Epoch 94
Loss = 1.6962e-02, PNorm = 83.6489, GNorm = 1.1727, lr_0 = 8.6198e-04
Validation prc-auc = 0.343293
Epoch 95
Validation prc-auc = 0.345032
Epoch 96
Validation prc-auc = 0.342256
Epoch 97
Validation prc-auc = 0.291128
Epoch 98
Validation prc-auc = 0.210684
Epoch 99
Loss = 1.9196e-02, PNorm = 83.8497, GNorm = 0.4856, lr_0 = 8.5461e-04
Validation prc-auc = 0.192574
Epoch 100
Validation prc-auc = 0.209601
Epoch 101
Validation prc-auc = 0.250479
Epoch 102
Validation prc-auc = 0.278383
Epoch 103
Validation prc-auc = 0.275290
Epoch 104
Loss = 7.1857e-03, PNorm = 84.0357, GNorm = 0.2142, lr_0 = 8.4730e-04
Validation prc-auc = 0.264635
Epoch 105
Validation prc-auc = 0.254282
Epoch 106
Validation prc-auc = 0.264551
Epoch 107
Validation prc-auc = 0.294027
Epoch 108
Validation prc-auc = 0.286518
Epoch 109
Loss = 9.0730e-03, PNorm = 84.1814, GNorm = 0.8205, lr_0 = 8.4006e-04
Validation prc-auc = 0.270109
Epoch 110
Validation prc-auc = 0.268162
Epoch 111
Validation prc-auc = 0.259521
Epoch 112
Validation prc-auc = 0.205277
Epoch 113
Validation prc-auc = 0.166000
Epoch 114
Loss = 4.9406e-02, PNorm = 84.3463, GNorm = 1.9243, lr_0 = 8.3288e-04
Validation prc-auc = 0.212214
Epoch 115
Validation prc-auc = 0.263161
Epoch 116
Validation prc-auc = 0.279538
Epoch 117
Validation prc-auc = 0.219397
Epoch 118
Validation prc-auc = 0.253764
Epoch 119
Loss = 2.6383e-02, PNorm = 84.5484, GNorm = 0.2824, lr_0 = 8.2576e-04
Validation prc-auc = 0.273885
Epoch 120
Validation prc-auc = 0.246262
Epoch 121
Validation prc-auc = 0.191422
Epoch 122
Validation prc-auc = 0.216522
Epoch 123
Validation prc-auc = 0.219966
Epoch 124
Loss = 6.1804e-02, PNorm = 84.7972, GNorm = 1.0151, lr_0 = 8.1870e-04
Validation prc-auc = 0.185787
Epoch 125
Validation prc-auc = 0.200373
Epoch 126
Validation prc-auc = 0.242191
Epoch 127
Validation prc-auc = 0.240296
Epoch 128
Validation prc-auc = 0.178176
Epoch 129
Loss = 3.3523e-02, PNorm = 85.1395, GNorm = 0.5256, lr_0 = 8.1170e-04
Validation prc-auc = 0.204151
Epoch 130
Validation prc-auc = 0.218663
Epoch 131
Validation prc-auc = 0.236722
Epoch 132
Validation prc-auc = 0.215811
Epoch 133
Validation prc-auc = 0.202910
Epoch 134
Loss = 2.1253e-02, PNorm = 85.4255, GNorm = 0.4560, lr_0 = 8.0476e-04
Validation prc-auc = 0.196777
Epoch 135
Validation prc-auc = 0.205130
Epoch 136
Validation prc-auc = 0.227541
Epoch 137
Validation prc-auc = 0.252211
Epoch 138
Validation prc-auc = 0.245700
Epoch 139
Loss = 1.3113e-02, PNorm = 85.6583, GNorm = 0.0394, lr_0 = 7.9788e-04
Validation prc-auc = 0.222978
Epoch 140
Validation prc-auc = 0.207730
Epoch 141
Validation prc-auc = 0.198344
Epoch 142
Validation prc-auc = 0.196818
Epoch 143
Validation prc-auc = 0.201621
Epoch 144
Loss = 9.6422e-03, PNorm = 85.8139, GNorm = 0.3940, lr_0 = 7.9106e-04
Validation prc-auc = 0.215057
Epoch 145
Validation prc-auc = 0.227365
Epoch 146
Validation prc-auc = 0.252091
Epoch 147
Validation prc-auc = 0.275157
Epoch 148
Validation prc-auc = 0.289513
Epoch 149
Loss = 7.7328e-03, PNorm = 85.9599, GNorm = 0.0524, lr_0 = 7.8430e-04
Validation prc-auc = 0.274051
Epoch 150
Validation prc-auc = 0.253683
Epoch 151
Validation prc-auc = 0.198872
Epoch 152
Validation prc-auc = 0.189283
Epoch 153
Validation prc-auc = 0.191910
Epoch 154
Loss = 7.6692e-02, PNorm = 86.0930, GNorm = 1.6271, lr_0 = 7.7759e-04
Validation prc-auc = 0.228123
Epoch 155
Validation prc-auc = 0.272988
Epoch 156
Validation prc-auc = 0.286564
Epoch 157
Validation prc-auc = 0.280947
Epoch 158
Validation prc-auc = 0.238652
Epoch 159
Loss = 1.5534e-03, PNorm = 86.2253, GNorm = 0.0538, lr_0 = 7.7095e-04
Validation prc-auc = 0.203787
Epoch 160
Validation prc-auc = 0.174199
Epoch 161
Validation prc-auc = 0.165425
Epoch 162
Validation prc-auc = 0.167142
Epoch 163
Validation prc-auc = 0.173436
Epoch 164
Loss = 5.2604e-03, PNorm = 86.3228, GNorm = 0.0879, lr_0 = 7.6436e-04
Validation prc-auc = 0.182415
Epoch 165
Validation prc-auc = 0.194889
Epoch 166
Validation prc-auc = 0.218257
Epoch 167
Validation prc-auc = 0.238327
Epoch 168
Validation prc-auc = 0.247811
Epoch 169
Loss = 6.8510e-03, PNorm = 86.4010, GNorm = 0.3714, lr_0 = 7.5782e-04
Validation prc-auc = 0.245389
Epoch 170
Validation prc-auc = 0.243590
Epoch 171
Validation prc-auc = 0.238518
Epoch 172
Validation prc-auc = 0.241509
Epoch 173
Validation prc-auc = 0.232566
Epoch 174
Loss = 6.0077e-04, PNorm = 86.4580, GNorm = 0.0183, lr_0 = 7.5134e-04
Validation prc-auc = 0.228940
Epoch 175
Validation prc-auc = 0.230387
Epoch 176
Validation prc-auc = 0.217262
Epoch 177
Validation prc-auc = 0.226027
Epoch 178
Validation prc-auc = 0.234933
Epoch 179
Loss = 1.2234e-02, PNorm = 86.5019, GNorm = 0.3012, lr_0 = 7.4492e-04
Validation prc-auc = 0.237254
Epoch 180
Validation prc-auc = 0.231102
Epoch 181
Validation prc-auc = 0.231997
Epoch 182
Validation prc-auc = 0.225636
Epoch 183
Validation prc-auc = 0.225364
Epoch 184
Loss = 1.1444e-03, PNorm = 86.5539, GNorm = 0.0869, lr_0 = 7.3855e-04
Validation prc-auc = 0.225363
Epoch 185
Validation prc-auc = 0.220254
Epoch 186
Validation prc-auc = 0.226408
Epoch 187
Validation prc-auc = 0.230151
Epoch 188
Validation prc-auc = 0.243914
Epoch 189
Loss = 1.5982e-03, PNorm = 86.5929, GNorm = 0.0528, lr_0 = 7.3224e-04
Validation prc-auc = 0.244414
Epoch 190
Validation prc-auc = 0.243082
Epoch 191
Validation prc-auc = 0.243138
Epoch 192
Validation prc-auc = 0.255176
Epoch 193
Validation prc-auc = 0.255786
Epoch 194
Loss = 7.6292e-04, PNorm = 86.6298, GNorm = 0.0353, lr_0 = 7.2598e-04
Validation prc-auc = 0.257413
Epoch 195
Validation prc-auc = 0.245336
Epoch 196
Validation prc-auc = 0.245865
Epoch 197
Validation prc-auc = 0.245849
Epoch 198
Validation prc-auc = 0.251351
Epoch 199
Loss = 4.9669e-04, PNorm = 86.6535, GNorm = 0.0209, lr_0 = 7.1977e-04
Validation prc-auc = 0.251486
Epoch 200
Validation prc-auc = 0.252000
Epoch 201
Validation prc-auc = 0.246061
Epoch 202
Validation prc-auc = 0.246128
Epoch 203
Validation prc-auc = 0.246128
Epoch 204
Loss = 2.3363e-04, PNorm = 86.6683, GNorm = 0.0039, lr_0 = 7.1362e-04
Validation prc-auc = 0.246311
Epoch 205
Validation prc-auc = 0.246404
Epoch 206
Validation prc-auc = 0.246159
Epoch 207
Validation prc-auc = 0.246159
Epoch 208
Validation prc-auc = 0.246227
Epoch 209
Loss = 3.2975e-04, PNorm = 86.6775, GNorm = 0.0150, lr_0 = 7.0752e-04
Validation prc-auc = 0.245827
Epoch 210
Validation prc-auc = 0.243924
Epoch 211
Validation prc-auc = 0.243597
Epoch 212
Validation prc-auc = 0.243450
Epoch 213
Validation prc-auc = 0.243279
Epoch 214
Loss = 3.8837e-04, PNorm = 86.6858, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.238650
Epoch 215
Validation prc-auc = 0.238392
Epoch 216
Validation prc-auc = 0.238233
Epoch 217
Validation prc-auc = 0.237587
Epoch 218
Validation prc-auc = 0.237032
Epoch 219
Loss = 4.7102e-04, PNorm = 86.6975, GNorm = 0.0065, lr_0 = 6.9548e-04
Validation prc-auc = 0.236958
Epoch 220
Validation prc-auc = 0.236481
Epoch 221
Validation prc-auc = 0.236329
Epoch 222
Validation prc-auc = 0.236706
Epoch 223
Validation prc-auc = 0.236978
Epoch 224
Loss = 1.0850e-04, PNorm = 86.7101, GNorm = 0.0023, lr_0 = 6.8953e-04
Validation prc-auc = 0.241532
Epoch 225
Validation prc-auc = 0.241425
Epoch 226
Validation prc-auc = 0.241691
Epoch 227
Validation prc-auc = 0.241903
Epoch 228
Validation prc-auc = 0.241592
Epoch 229
Loss = 1.9321e-04, PNorm = 86.7197, GNorm = 0.0052, lr_0 = 6.8364e-04
Validation prc-auc = 0.241513
Epoch 230
Validation prc-auc = 0.241515
Epoch 231
Validation prc-auc = 0.241367
Epoch 232
Validation prc-auc = 0.240913
Epoch 233
Validation prc-auc = 0.241020
Epoch 234
Loss = 1.1889e-04, PNorm = 86.7272, GNorm = 0.0030, lr_0 = 6.7779e-04
Validation prc-auc = 0.241020
Epoch 235
Validation prc-auc = 0.241082
Epoch 236
Validation prc-auc = 0.240905
Epoch 237
Validation prc-auc = 0.240910
Epoch 238
Validation prc-auc = 0.240509
Epoch 239
Loss = 1.9137e-04, PNorm = 86.7349, GNorm = 0.0056, lr_0 = 6.7200e-04
Validation prc-auc = 0.240506
Epoch 240
Validation prc-auc = 0.240262
Epoch 241
Validation prc-auc = 0.240089
Epoch 242
Validation prc-auc = 0.240016
Epoch 243
Validation prc-auc = 0.240016
Epoch 244
Loss = 1.2605e-04, PNorm = 86.7433, GNorm = 0.0025, lr_0 = 6.6625e-04
Validation prc-auc = 0.246301
Epoch 245
Validation prc-auc = 0.246373
Epoch 246
Validation prc-auc = 0.240089
Epoch 247
Validation prc-auc = 0.240089
Epoch 248
Validation prc-auc = 0.239977
Epoch 249
Loss = 1.0577e-04, PNorm = 86.7513, GNorm = 0.0024, lr_0 = 6.6056e-04
Validation prc-auc = 0.240178
Epoch 250
Validation prc-auc = 0.240178
Epoch 251
Validation prc-auc = 0.240176
Epoch 252
Validation prc-auc = 0.239800
Epoch 253
Validation prc-auc = 0.240341
Epoch 254
Loss = 4.3154e-05, PNorm = 86.7612, GNorm = 0.0018, lr_0 = 6.5491e-04
Validation prc-auc = 0.240416
Epoch 255
Validation prc-auc = 0.240238
Epoch 256
Validation prc-auc = 0.240495
Epoch 257
Validation prc-auc = 0.240490
Epoch 258
Validation prc-auc = 0.240557
Epoch 259
Loss = 5.0080e-05, PNorm = 86.7691, GNorm = 0.0008, lr_0 = 6.4931e-04
Validation prc-auc = 0.240635
Epoch 260
Validation prc-auc = 0.240310
Epoch 261
Validation prc-auc = 0.240373
Epoch 262
Validation prc-auc = 0.240293
Epoch 263
Validation prc-auc = 0.240440
Epoch 264
Loss = 5.8278e-05, PNorm = 86.7764, GNorm = 0.0018, lr_0 = 6.4376e-04
Validation prc-auc = 0.240860
Epoch 265
Validation prc-auc = 0.241244
Epoch 266
Validation prc-auc = 0.241244
Epoch 267
Validation prc-auc = 0.241372
Epoch 268
Validation prc-auc = 0.241298
Epoch 269
Loss = 1.0803e-04, PNorm = 86.7852, GNorm = 0.0031, lr_0 = 6.3826e-04
Validation prc-auc = 0.241465
Epoch 270
Validation prc-auc = 0.241605
Epoch 271
Validation prc-auc = 0.241605
Epoch 272
Validation prc-auc = 0.241691
Epoch 273
Validation prc-auc = 0.241691
Epoch 274
Loss = 1.1218e-04, PNorm = 86.7927, GNorm = 0.0035, lr_0 = 6.3280e-04
Validation prc-auc = 0.241691
Epoch 275
Validation prc-auc = 0.243121
Epoch 276
Validation prc-auc = 0.243121
Epoch 277
Validation prc-auc = 0.242998
Epoch 278
Validation prc-auc = 0.242833
Epoch 279
Loss = 1.2764e-04, PNorm = 86.8023, GNorm = 0.0025, lr_0 = 6.2739e-04
Validation prc-auc = 0.243086
Epoch 280
Validation prc-auc = 0.243345
Epoch 281
Validation prc-auc = 0.243438
Epoch 282
Validation prc-auc = 0.243802
Epoch 283
Validation prc-auc = 0.243802
Epoch 284
Loss = 1.4733e-04, PNorm = 86.8134, GNorm = 0.0133, lr_0 = 6.2203e-04
Validation prc-auc = 0.243856
Epoch 285
Validation prc-auc = 0.243856
Epoch 286
Validation prc-auc = 0.244168
Epoch 287
Validation prc-auc = 0.244098
Epoch 288
Validation prc-auc = 0.244098
Epoch 289
Loss = 3.4529e-05, PNorm = 86.8239, GNorm = 0.0006, lr_0 = 6.1671e-04
Validation prc-auc = 0.243966
Epoch 290
Validation prc-auc = 0.243693
Epoch 291
Validation prc-auc = 0.243623
Epoch 292
Validation prc-auc = 0.243940
Epoch 293
Validation prc-auc = 0.244010
Epoch 294
Loss = 3.3168e-05, PNorm = 86.8315, GNorm = 0.0009, lr_0 = 6.1144e-04
Validation prc-auc = 0.250264
Epoch 295
Validation prc-auc = 0.249965
Epoch 296
Validation prc-auc = 0.249854
Epoch 297
Validation prc-auc = 0.249821
Epoch 298
Validation prc-auc = 0.249514
Epoch 299
Loss = 3.6395e-05, PNorm = 86.8380, GNorm = 0.0027, lr_0 = 6.0621e-04
Validation prc-auc = 0.249688
Model 0 best validation prc-auc = 0.524092 on epoch 26
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.512151
Ensemble test prc-auc = 0.512151
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.451981
Epoch 1
Validation prc-auc = 0.446994
Epoch 2
Validation prc-auc = 0.456972
Epoch 3
Validation prc-auc = 0.434877
Epoch 4
Loss = 7.3300e-01, PNorm = 78.5513, GNorm = 15.6126, lr_0 = 6.5000e-04
Validation prc-auc = 0.447210
Epoch 5
Validation prc-auc = 0.363943
Epoch 6
Validation prc-auc = 0.448088
Epoch 7
Validation prc-auc = 0.492841
Epoch 8
Validation prc-auc = 0.466519
Epoch 9
Loss = 6.7786e-01, PNorm = 78.8123, GNorm = 6.3473, lr_0 = 9.9743e-04
Validation prc-auc = 0.485870
Epoch 10
Validation prc-auc = 0.319707
Epoch 11
Validation prc-auc = 0.311744
Epoch 12
Validation prc-auc = 0.295678
Epoch 13
Validation prc-auc = 0.431364
Epoch 14
Loss = 5.5333e-01, PNorm = 79.1009, GNorm = 3.2639, lr_0 = 9.8890e-04
Validation prc-auc = 0.291894
Epoch 15
Validation prc-auc = 0.298993
Epoch 16
Validation prc-auc = 0.320662
Epoch 17
Validation prc-auc = 0.412705
Epoch 18
Validation prc-auc = 0.392252
Epoch 19
Loss = 4.9499e-01, PNorm = 79.3513, GNorm = 1.1391, lr_0 = 9.8045e-04
Validation prc-auc = 0.398266
Epoch 20
Validation prc-auc = 0.415484
Epoch 21
Validation prc-auc = 0.302287
Epoch 22
Validation prc-auc = 0.241544
Epoch 23
Validation prc-auc = 0.332738
Epoch 24
Loss = 4.7328e-01, PNorm = 79.6106, GNorm = 4.8199, lr_0 = 9.7207e-04
Validation prc-auc = 0.361985
Epoch 25
Validation prc-auc = 0.338897
Epoch 26
Validation prc-auc = 0.362187
Epoch 27
Validation prc-auc = 0.343060
Epoch 28
Validation prc-auc = 0.315752
Epoch 29
Loss = 4.4372e-01, PNorm = 79.9035, GNorm = 5.2715, lr_0 = 9.6376e-04
Validation prc-auc = 0.294318
Epoch 30
Validation prc-auc = 0.390667
Epoch 31
Validation prc-auc = 0.411278
Epoch 32
Validation prc-auc = 0.419696
Epoch 33
Validation prc-auc = 0.427415
Epoch 34
Loss = 4.3009e-01, PNorm = 80.2022, GNorm = 3.1468, lr_0 = 9.5552e-04
Validation prc-auc = 0.421484
Epoch 35
Validation prc-auc = 0.332690
Epoch 36
Validation prc-auc = 0.343598
Epoch 37
Validation prc-auc = 0.348775
Epoch 38
Validation prc-auc = 0.382581
Epoch 39
Loss = 3.1726e-01, PNorm = 80.4962, GNorm = 0.8414, lr_0 = 9.4735e-04
Validation prc-auc = 0.381942
Epoch 40
Validation prc-auc = 0.362318
Epoch 41
Validation prc-auc = 0.355381
Epoch 42
Validation prc-auc = 0.322578
Epoch 43
Validation prc-auc = 0.332220
Epoch 44
Loss = 2.5246e-01, PNorm = 80.8630, GNorm = 1.3613, lr_0 = 9.3925e-04
Validation prc-auc = 0.316774
Epoch 45
Validation prc-auc = 0.297024
Epoch 46
Validation prc-auc = 0.283733
Epoch 47
Validation prc-auc = 0.291580
Epoch 48
Validation prc-auc = 0.286660
Epoch 49
Loss = 1.8035e-01, PNorm = 81.2976, GNorm = 1.0920, lr_0 = 9.3122e-04
Validation prc-auc = 0.272602
Epoch 50
Validation prc-auc = 0.287387
Epoch 51
Validation prc-auc = 0.266252
Epoch 52
Validation prc-auc = 0.163727
Epoch 53
Validation prc-auc = 0.213129
Epoch 54
Loss = 1.4254e-01, PNorm = 81.7439, GNorm = 0.5898, lr_0 = 9.2326e-04
Validation prc-auc = 0.231894
Epoch 55
Validation prc-auc = 0.380404
Epoch 56
Validation prc-auc = 0.350888
Epoch 57
Validation prc-auc = 0.358191
Epoch 58
Validation prc-auc = 0.363654
Epoch 59
Loss = 1.2802e-01, PNorm = 82.1729, GNorm = 1.3726, lr_0 = 9.1537e-04
Validation prc-auc = 0.296677
Epoch 60
Validation prc-auc = 0.280951
Epoch 61
Validation prc-auc = 0.272732
Epoch 62
Validation prc-auc = 0.256215
Epoch 63
Validation prc-auc = 0.261775
Epoch 64
Loss = 6.7330e-02, PNorm = 82.5714, GNorm = 1.0194, lr_0 = 9.0754e-04
Validation prc-auc = 0.266497
Epoch 65
Validation prc-auc = 0.302952
Epoch 66
Validation prc-auc = 0.313754
Epoch 67
Validation prc-auc = 0.313195
Epoch 68
Validation prc-auc = 0.331574
Epoch 69
Loss = 9.0878e-02, PNorm = 82.9257, GNorm = 2.1943, lr_0 = 8.9978e-04
Validation prc-auc = 0.300863
Epoch 70
Validation prc-auc = 0.286141
Epoch 71
Validation prc-auc = 0.296993
Epoch 72
Validation prc-auc = 0.281545
Epoch 73
Validation prc-auc = 0.276575
Epoch 74
Loss = 2.3740e-02, PNorm = 83.2514, GNorm = 0.3164, lr_0 = 8.9209e-04
Validation prc-auc = 0.282224
Epoch 75
Validation prc-auc = 0.279330
Epoch 76
Validation prc-auc = 0.263277
Epoch 77
Validation prc-auc = 0.255019
Epoch 78
Validation prc-auc = 0.271515
Epoch 79
Loss = 4.1753e-02, PNorm = 83.5512, GNorm = 0.4995, lr_0 = 8.8447e-04
Validation prc-auc = 0.315346
Epoch 80
Validation prc-auc = 0.330518
Epoch 81
Validation prc-auc = 0.306628
Epoch 82
Validation prc-auc = 0.302877
Epoch 83
Validation prc-auc = 0.333172
Epoch 84
Loss = 1.0037e-02, PNorm = 83.8114, GNorm = 0.1544, lr_0 = 8.7691e-04
Validation prc-auc = 0.332299
Epoch 85
Validation prc-auc = 0.334477
Epoch 86
Validation prc-auc = 0.329965
Epoch 87
Validation prc-auc = 0.270618
Epoch 88
Validation prc-auc = 0.278125
Epoch 89
Loss = 1.0980e-01, PNorm = 84.0138, GNorm = 1.2087, lr_0 = 8.6941e-04
Validation prc-auc = 0.307675
Epoch 90
Validation prc-auc = 0.331740
Epoch 91
Validation prc-auc = 0.351743
Epoch 92
Validation prc-auc = 0.350187
Epoch 93
Validation prc-auc = 0.318731
Epoch 94
Loss = 1.5660e-01, PNorm = 84.2226, GNorm = 1.2927, lr_0 = 8.6198e-04
Validation prc-auc = 0.326551
Epoch 95
Validation prc-auc = 0.291725
Epoch 96
Validation prc-auc = 0.195668
Epoch 97
Validation prc-auc = 0.184707
Epoch 98
Validation prc-auc = 0.232739
Epoch 99
Loss = 6.2176e-02, PNorm = 84.4634, GNorm = 2.4517, lr_0 = 8.5461e-04
Validation prc-auc = 0.248480
Epoch 100
Validation prc-auc = 0.179112
Epoch 101
Validation prc-auc = 0.201404
Epoch 102
Validation prc-auc = 0.247289
Epoch 103
Validation prc-auc = 0.251840
Epoch 104
Loss = 3.0495e-02, PNorm = 84.7787, GNorm = 2.0164, lr_0 = 8.4730e-04
Validation prc-auc = 0.284531
Epoch 105
Validation prc-auc = 0.336537
Epoch 106
Validation prc-auc = 0.356780
Epoch 107
Validation prc-auc = 0.359496
Epoch 108
Validation prc-auc = 0.332561
Epoch 109
Loss = 2.3595e-02, PNorm = 85.1265, GNorm = 0.2145, lr_0 = 8.4006e-04
Validation prc-auc = 0.289069
Epoch 110
Validation prc-auc = 0.262988
Epoch 111
Validation prc-auc = 0.271154
Epoch 112
Validation prc-auc = 0.264520
Epoch 113
Validation prc-auc = 0.269497
Epoch 114
Loss = 2.2038e-01, PNorm = 85.4287, GNorm = 2.4526, lr_0 = 8.3288e-04
Validation prc-auc = 0.244269
Epoch 115
Validation prc-auc = 0.184818
Epoch 116
Validation prc-auc = 0.165806
Epoch 117
Validation prc-auc = 0.254500
Epoch 118
Validation prc-auc = 0.312299
Epoch 119
Loss = 5.2687e-02, PNorm = 85.7742, GNorm = 0.7315, lr_0 = 8.2576e-04
Validation prc-auc = 0.274522
Epoch 120
Validation prc-auc = 0.226121
Epoch 121
Validation prc-auc = 0.215974
Epoch 122
Validation prc-auc = 0.204843
Epoch 123
Validation prc-auc = 0.243778
Epoch 124
Loss = 6.1667e-02, PNorm = 86.0668, GNorm = 2.7278, lr_0 = 8.1870e-04
Validation prc-auc = 0.215740
Epoch 125
Validation prc-auc = 0.204278
Epoch 126
Validation prc-auc = 0.237765
Epoch 127
Validation prc-auc = 0.236781
Epoch 128
Validation prc-auc = 0.202069
Epoch 129
Loss = 3.9290e-02, PNorm = 86.4271, GNorm = 1.4355, lr_0 = 8.1170e-04
Validation prc-auc = 0.171611
Epoch 130
Validation prc-auc = 0.188763
Epoch 131
Validation prc-auc = 0.273622
Epoch 132
Validation prc-auc = 0.285016
Epoch 133
Validation prc-auc = 0.295705
Epoch 134
Loss = 2.2487e-02, PNorm = 86.7805, GNorm = 0.6677, lr_0 = 8.0476e-04
Validation prc-auc = 0.295033
Epoch 135
Validation prc-auc = 0.313064
Epoch 136
Validation prc-auc = 0.314268
Epoch 137
Validation prc-auc = 0.305511
Epoch 138
Validation prc-auc = 0.300970
Epoch 139
Loss = 1.4917e-02, PNorm = 87.0262, GNorm = 0.8027, lr_0 = 7.9788e-04
Validation prc-auc = 0.289984
Epoch 140
Validation prc-auc = 0.289688
Epoch 141
Validation prc-auc = 0.272827
Epoch 142
Validation prc-auc = 0.243509
Epoch 143
Validation prc-auc = 0.229837
Epoch 144
Loss = 1.1503e-02, PNorm = 87.2025, GNorm = 0.1404, lr_0 = 7.9106e-04
Validation prc-auc = 0.200323
Epoch 145
Validation prc-auc = 0.179604
Epoch 146
Validation prc-auc = 0.177672
Epoch 147
Validation prc-auc = 0.190052
Epoch 148
Validation prc-auc = 0.203944
Epoch 149
Loss = 1.1095e-02, PNorm = 87.3288, GNorm = 0.3025, lr_0 = 7.8430e-04
Validation prc-auc = 0.204917
Epoch 150
Validation prc-auc = 0.209018
Epoch 151
Validation prc-auc = 0.209511
Epoch 152
Validation prc-auc = 0.204461
Epoch 153
Validation prc-auc = 0.237384
Epoch 154
Loss = 3.4870e-03, PNorm = 87.4270, GNorm = 0.1493, lr_0 = 7.7759e-04
Validation prc-auc = 0.246913
Epoch 155
Validation prc-auc = 0.250661
Epoch 156
Validation prc-auc = 0.259806
Epoch 157
Validation prc-auc = 0.263446
Epoch 158
Validation prc-auc = 0.269533
Epoch 159
Loss = 2.8702e-03, PNorm = 87.5061, GNorm = 0.0384, lr_0 = 7.7095e-04
Validation prc-auc = 0.274294
Epoch 160
Validation prc-auc = 0.261164
Epoch 161
Validation prc-auc = 0.260509
Epoch 162
Validation prc-auc = 0.257529
Epoch 163
Validation prc-auc = 0.237820
Epoch 164
Loss = 2.2486e-03, PNorm = 87.5652, GNorm = 0.0561, lr_0 = 7.6436e-04
Validation prc-auc = 0.234714
Epoch 165
Validation prc-auc = 0.223052
Epoch 166
Validation prc-auc = 0.230105
Epoch 167
Validation prc-auc = 0.230004
Epoch 168
Validation prc-auc = 0.226148
Epoch 169
Loss = 1.4940e-03, PNorm = 87.6145, GNorm = 0.0345, lr_0 = 7.5782e-04
Validation prc-auc = 0.225737
Epoch 170
Validation prc-auc = 0.217345
Epoch 171
Validation prc-auc = 0.217235
Epoch 172
Validation prc-auc = 0.217558
Epoch 173
Validation prc-auc = 0.223173
Epoch 174
Loss = 8.7847e-04, PNorm = 87.6570, GNorm = 0.0279, lr_0 = 7.5134e-04
Validation prc-auc = 0.222471
Epoch 175
Validation prc-auc = 0.222174
Epoch 176
Validation prc-auc = 0.220916
Epoch 177
Validation prc-auc = 0.220301
Epoch 178
Validation prc-auc = 0.220193
Epoch 179
Loss = 4.9069e-04, PNorm = 87.6854, GNorm = 0.0094, lr_0 = 7.4492e-04
Validation prc-auc = 0.219958
Epoch 180
Validation prc-auc = 0.219895
Epoch 181
Validation prc-auc = 0.221900
Epoch 182
Validation prc-auc = 0.222981
Epoch 183
Validation prc-auc = 0.231372
Epoch 184
Loss = 1.1542e-03, PNorm = 87.7007, GNorm = 0.0182, lr_0 = 7.3855e-04
Validation prc-auc = 0.233313
Epoch 185
Validation prc-auc = 0.240918
Epoch 186
Validation prc-auc = 0.242466
Epoch 187
Validation prc-auc = 0.242526
Epoch 188
Validation prc-auc = 0.244635
Epoch 189
Loss = 2.0666e-03, PNorm = 87.7300, GNorm = 0.0243, lr_0 = 7.3224e-04
Validation prc-auc = 0.252570
Epoch 190
Validation prc-auc = 0.265322
Epoch 191
Validation prc-auc = 0.261996
Epoch 192
Validation prc-auc = 0.264496
Epoch 193
Validation prc-auc = 0.255808
Epoch 194
Loss = 4.4080e-03, PNorm = 87.7632, GNorm = 0.4909, lr_0 = 7.2598e-04
Validation prc-auc = 0.243308
Epoch 195
Validation prc-auc = 0.249994
Epoch 196
Validation prc-auc = 0.238423
Epoch 197
Validation prc-auc = 0.234646
Epoch 198
Validation prc-auc = 0.247178
Epoch 199
Loss = 7.2347e-04, PNorm = 87.7936, GNorm = 0.0180, lr_0 = 7.1977e-04
Validation prc-auc = 0.246259
Epoch 200
Validation prc-auc = 0.251045
Epoch 201
Validation prc-auc = 0.257668
Epoch 202
Validation prc-auc = 0.257949
Epoch 203
Validation prc-auc = 0.260627
Epoch 204
Loss = 7.0931e-04, PNorm = 87.8198, GNorm = 0.0246, lr_0 = 7.1362e-04
Validation prc-auc = 0.261541
Epoch 205
Validation prc-auc = 0.261455
Epoch 206
Validation prc-auc = 0.264477
Epoch 207
Validation prc-auc = 0.262961
Epoch 208
Validation prc-auc = 0.263147
Epoch 209
Loss = 4.5923e-04, PNorm = 87.8424, GNorm = 0.0146, lr_0 = 7.0752e-04
Validation prc-auc = 0.260400
Epoch 210
Validation prc-auc = 0.254651
Epoch 211
Validation prc-auc = 0.254685
Epoch 212
Validation prc-auc = 0.254610
Epoch 213
Validation prc-auc = 0.255038
Epoch 214
Loss = 3.3965e-04, PNorm = 87.8602, GNorm = 0.0060, lr_0 = 7.0147e-04
Validation prc-auc = 0.255038
Epoch 215
Validation prc-auc = 0.254281
Epoch 216
Validation prc-auc = 0.253146
Epoch 217
Validation prc-auc = 0.253506
Epoch 218
Validation prc-auc = 0.254100
Epoch 219
Loss = 6.8978e-04, PNorm = 87.8762, GNorm = 0.0085, lr_0 = 6.9548e-04
Validation prc-auc = 0.252629
Epoch 220
Validation prc-auc = 0.261809
Epoch 221
Validation prc-auc = 0.268167
Epoch 222
Validation prc-auc = 0.268249
Epoch 223
Validation prc-auc = 0.268976
Epoch 224
Loss = 4.5019e-04, PNorm = 87.8891, GNorm = 0.0198, lr_0 = 6.8953e-04
Validation prc-auc = 0.269516
Epoch 225
Validation prc-auc = 0.274278
Epoch 226
Validation prc-auc = 0.274109
Epoch 227
Validation prc-auc = 0.273516
Epoch 228
Validation prc-auc = 0.265833
Epoch 229
Loss = 1.7040e-04, PNorm = 87.9016, GNorm = 0.0061, lr_0 = 6.8364e-04
Validation prc-auc = 0.270684
Epoch 230
Validation prc-auc = 0.270185
Epoch 231
Validation prc-auc = 0.270694
Epoch 232
Validation prc-auc = 0.270751
Epoch 233
Validation prc-auc = 0.270631
Epoch 234
Loss = 1.4315e-04, PNorm = 87.9119, GNorm = 0.0033, lr_0 = 6.7779e-04
Validation prc-auc = 0.270455
Epoch 235
Validation prc-auc = 0.270370
Epoch 236
Validation prc-auc = 0.270231
Epoch 237
Validation prc-auc = 0.270231
Epoch 238
Validation prc-auc = 0.270231
Epoch 239
Loss = 1.9387e-04, PNorm = 87.9192, GNorm = 0.0094, lr_0 = 6.7200e-04
Validation prc-auc = 0.269676
Epoch 240
Validation prc-auc = 0.269793
Epoch 241
Validation prc-auc = 0.269793
Epoch 242
Validation prc-auc = 0.269793
Epoch 243
Validation prc-auc = 0.269793
Epoch 244
Loss = 1.6600e-04, PNorm = 87.9262, GNorm = 0.0035, lr_0 = 6.6625e-04
Validation prc-auc = 0.269894
Epoch 245
Validation prc-auc = 0.270105
Epoch 246
Validation prc-auc = 0.276827
Epoch 247
Validation prc-auc = 0.276363
Epoch 248
Validation prc-auc = 0.276339
Epoch 249
Loss = 1.9239e-04, PNorm = 87.9332, GNorm = 0.0101, lr_0 = 6.6056e-04
Validation prc-auc = 0.275668
Epoch 250
Validation prc-auc = 0.275034
Epoch 251
Validation prc-auc = 0.268477
Epoch 252
Validation prc-auc = 0.268418
Epoch 253
Validation prc-auc = 0.268747
Epoch 254
Loss = 1.3926e-04, PNorm = 87.9405, GNorm = 0.0027, lr_0 = 6.5491e-04
Validation prc-auc = 0.268457
Epoch 255
Validation prc-auc = 0.268238
Epoch 256
Validation prc-auc = 0.268125
Epoch 257
Validation prc-auc = 0.268185
Epoch 258
Validation prc-auc = 0.275545
Epoch 259
Loss = 9.2985e-05, PNorm = 87.9478, GNorm = 0.0034, lr_0 = 6.4931e-04
Validation prc-auc = 0.275510
Epoch 260
Validation prc-auc = 0.275650
Epoch 261
Validation prc-auc = 0.276198
Epoch 262
Validation prc-auc = 0.276275
Epoch 263
Validation prc-auc = 0.276275
Epoch 264
Loss = 1.4267e-04, PNorm = 87.9554, GNorm = 0.0052, lr_0 = 6.4376e-04
Validation prc-auc = 0.276192
Epoch 265
Validation prc-auc = 0.276192
Epoch 266
Validation prc-auc = 0.276054
Epoch 267
Validation prc-auc = 0.276054
Epoch 268
Validation prc-auc = 0.276232
Epoch 269
Loss = 1.1702e-04, PNorm = 87.9637, GNorm = 0.0053, lr_0 = 6.3826e-04
Validation prc-auc = 0.276206
Epoch 270
Validation prc-auc = 0.276298
Epoch 271
Validation prc-auc = 0.276481
Epoch 272
Validation prc-auc = 0.276481
Epoch 273
Validation prc-auc = 0.276481
Epoch 274
Loss = 3.3947e-05, PNorm = 87.9706, GNorm = 0.0006, lr_0 = 6.3280e-04
Validation prc-auc = 0.276481
Epoch 275
Validation prc-auc = 0.276337
Epoch 276
Validation prc-auc = 0.276512
Epoch 277
Validation prc-auc = 0.276575
Epoch 278
Validation prc-auc = 0.275938
Epoch 279
Loss = 6.3084e-05, PNorm = 87.9764, GNorm = 0.0055, lr_0 = 6.2739e-04
Validation prc-auc = 0.275856
Epoch 280
Validation prc-auc = 0.275670
Epoch 281
Validation prc-auc = 0.275638
Epoch 282
Validation prc-auc = 0.264682
Epoch 283
Validation prc-auc = 0.265450
Epoch 284
Loss = 7.0341e-05, PNorm = 87.9810, GNorm = 0.0016, lr_0 = 6.2203e-04
Validation prc-auc = 0.265515
Epoch 285
Validation prc-auc = 0.265515
Epoch 286
Validation prc-auc = 0.265436
Epoch 287
Validation prc-auc = 0.275830
Epoch 288
Validation prc-auc = 0.275823
Epoch 289
Loss = 4.6313e-05, PNorm = 87.9878, GNorm = 0.0019, lr_0 = 6.1671e-04
Validation prc-auc = 0.275288
Epoch 290
Validation prc-auc = 0.274733
Epoch 291
Validation prc-auc = 0.274733
Epoch 292
Validation prc-auc = 0.274351
Epoch 293
Validation prc-auc = 0.274579
Epoch 294
Loss = 1.4626e-04, PNorm = 87.9938, GNorm = 0.0015, lr_0 = 6.1144e-04
Validation prc-auc = 0.274670
Epoch 295
Validation prc-auc = 0.274853
Epoch 296
Validation prc-auc = 0.275408
Epoch 297
Validation prc-auc = 0.276237
Epoch 298
Validation prc-auc = 0.277788
Epoch 299
Loss = 4.2091e-05, PNorm = 87.9985, GNorm = 0.0032, lr_0 = 6.0621e-04
Validation prc-auc = 0.277967
Model 0 best validation prc-auc = 0.492841 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.437878
Ensemble test prc-auc = 0.437878
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.05, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1800, bias=False)
        (W_h): Linear(in_features=1800, out_features=1800, bias=False)
        (W_o): Linear(in_features=1933, out_features=1800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.05, inplace=False)
    (1): Linear(in_features=2900, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.05, inplace=False)
    (4): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 12,209,401
Moving model to cuda
Epoch 0
Validation prc-auc = 0.447435
Epoch 1
Validation prc-auc = 0.455436
Epoch 2
Validation prc-auc = 0.437530
Epoch 3
Validation prc-auc = 0.433102
Epoch 4
Loss = 6.7931e-01, PNorm = 78.5478, GNorm = 5.8820, lr_0 = 6.5000e-04
Validation prc-auc = 0.440586
Epoch 5
Validation prc-auc = 0.497692
Epoch 6
Validation prc-auc = 0.424765
Epoch 7
Validation prc-auc = 0.395213
Epoch 8
Validation prc-auc = 0.294183
Epoch 9
Loss = 5.2405e-01, PNorm = 78.7933, GNorm = 6.3021, lr_0 = 9.9743e-04
Validation prc-auc = 0.335632
Epoch 10
Validation prc-auc = 0.283611
Epoch 11
Validation prc-auc = 0.289981
Epoch 12
Validation prc-auc = 0.306816
Epoch 13
Validation prc-auc = 0.280692
Epoch 14
Loss = 5.3578e-01, PNorm = 79.0958, GNorm = 5.6050, lr_0 = 9.8890e-04
Validation prc-auc = 0.455066
Epoch 15
Validation prc-auc = 0.476875
Epoch 16
Validation prc-auc = 0.488546
Epoch 17
Validation prc-auc = 0.270603
Epoch 18
Validation prc-auc = 0.275675
Epoch 19
Loss = 5.5998e-01, PNorm = 79.4052, GNorm = 11.0964, lr_0 = 9.8045e-04
Validation prc-auc = 0.270895
Epoch 20
Validation prc-auc = 0.401254
Epoch 21
Validation prc-auc = 0.414864
Epoch 22
Validation prc-auc = 0.423959
Epoch 23
Validation prc-auc = 0.415277
Epoch 24
Loss = 4.8958e-01, PNorm = 79.7725, GNorm = 2.0160, lr_0 = 9.7207e-04
Validation prc-auc = 0.295220
Epoch 25
Validation prc-auc = 0.323758
Epoch 26
Validation prc-auc = 0.364850
Epoch 27
Validation prc-auc = 0.338250
Epoch 28
Validation prc-auc = 0.334354
Epoch 29
Loss = 3.8850e-01, PNorm = 80.1224, GNorm = 3.6337, lr_0 = 9.6376e-04
Validation prc-auc = 0.319221
Epoch 30
Validation prc-auc = 0.292483
Epoch 31
Validation prc-auc = 0.276862
Epoch 32
Validation prc-auc = 0.329416
Epoch 33
Validation prc-auc = 0.317479
Epoch 34
Loss = 2.8719e-01, PNorm = 80.4957, GNorm = 1.0380, lr_0 = 9.5552e-04
Validation prc-auc = 0.358420
Epoch 35
Validation prc-auc = 0.317927
Epoch 36
Validation prc-auc = 0.296096
Epoch 37
Validation prc-auc = 0.275368
Epoch 38
Validation prc-auc = 0.245756
Epoch 39
Loss = 3.0297e-01, PNorm = 80.8953, GNorm = 1.1152, lr_0 = 9.4735e-04
Validation prc-auc = 0.261501
Epoch 40
Validation prc-auc = 0.315310
Epoch 41
Validation prc-auc = 0.302531
Epoch 42
Validation prc-auc = 0.318977
Epoch 43
Validation prc-auc = 0.310490
Epoch 44
Loss = 1.7789e-01, PNorm = 81.3788, GNorm = 2.3393, lr_0 = 9.3925e-04
Validation prc-auc = 0.288724
Epoch 45
Validation prc-auc = 0.273710
Epoch 46
Validation prc-auc = 0.309345
Epoch 47
Validation prc-auc = 0.295845
Epoch 48
Validation prc-auc = 0.303109
Epoch 49
Loss = 3.0189e-01, PNorm = 81.9124, GNorm = 3.0181, lr_0 = 9.3122e-04
Validation prc-auc = 0.320837
Epoch 50
Validation prc-auc = 0.324325
Epoch 51
Validation prc-auc = 0.312761
Epoch 52
Validation prc-auc = 0.308602
Epoch 53
Validation prc-auc = 0.256017
Epoch 54
Loss = 9.6265e-02, PNorm = 82.4519, GNorm = 1.2409, lr_0 = 9.2326e-04
Validation prc-auc = 0.263593
Epoch 55
Validation prc-auc = 0.248617
Epoch 56
Validation prc-auc = 0.235027
Epoch 57
Validation prc-auc = 0.304428
Epoch 58
Validation prc-auc = 0.309252
Epoch 59
Loss = 5.3974e-02, PNorm = 82.9363, GNorm = 0.8254, lr_0 = 9.1537e-04
Validation prc-auc = 0.307894
Epoch 60
Validation prc-auc = 0.308851
Epoch 61
Validation prc-auc = 0.295125
Epoch 62
Validation prc-auc = 0.274480
Epoch 63
Validation prc-auc = 0.265507
Epoch 64
Loss = 1.0481e-01, PNorm = 83.3483, GNorm = 0.3695, lr_0 = 9.0754e-04
Validation prc-auc = 0.220221
Epoch 65
Validation prc-auc = 0.294121
Epoch 66
Validation prc-auc = 0.297513
Epoch 67
Validation prc-auc = 0.274874
Epoch 68
Validation prc-auc = 0.244755
Epoch 69
Loss = 6.7708e-02, PNorm = 83.7869, GNorm = 2.4059, lr_0 = 8.9978e-04
Validation prc-auc = 0.242615
Epoch 70
Validation prc-auc = 0.242499
Epoch 71
Validation prc-auc = 0.254930
Epoch 72
Validation prc-auc = 0.275424
Epoch 73
Validation prc-auc = 0.268193
Epoch 74
Loss = 2.9544e-02, PNorm = 84.1494, GNorm = 0.5043, lr_0 = 8.9209e-04
Validation prc-auc = 0.267116
Epoch 75
Validation prc-auc = 0.281949
Epoch 76
Validation prc-auc = 0.276047
Epoch 77
Validation prc-auc = 0.205832
Epoch 78
Validation prc-auc = 0.183349
Epoch 79
Loss = 7.6917e-02, PNorm = 84.4472, GNorm = 2.5060, lr_0 = 8.8447e-04
Validation prc-auc = 0.253312
Epoch 80
Validation prc-auc = 0.269865
Epoch 81
Validation prc-auc = 0.248978
Epoch 82
Validation prc-auc = 0.212942
Epoch 83
Validation prc-auc = 0.272852
Epoch 84
Loss = 5.6095e-02, PNorm = 84.8400, GNorm = 1.7676, lr_0 = 8.7691e-04
Validation prc-auc = 0.270776
Epoch 85
Validation prc-auc = 0.279112
Epoch 86
Validation prc-auc = 0.261560
Epoch 87
Validation prc-auc = 0.245093
Epoch 88
Validation prc-auc = 0.255377
Epoch 89
Loss = 2.8817e-02, PNorm = 85.1837, GNorm = 0.2151, lr_0 = 8.6941e-04
Validation prc-auc = 0.269885
Epoch 90
Validation prc-auc = 0.274003
Epoch 91
Validation prc-auc = 0.263056
Epoch 92
Validation prc-auc = 0.258085
Epoch 93
Validation prc-auc = 0.258102
Epoch 94
Loss = 1.4169e-02, PNorm = 85.4474, GNorm = 0.4876, lr_0 = 8.6198e-04
Validation prc-auc = 0.253735
Epoch 95
Validation prc-auc = 0.248421
Epoch 96
Validation prc-auc = 0.236368
Epoch 97
Validation prc-auc = 0.212348
Epoch 98
Validation prc-auc = 0.202274
Epoch 99
Loss = 1.1080e-02, PNorm = 85.6194, GNorm = 0.1793, lr_0 = 8.5461e-04
Validation prc-auc = 0.210346
Epoch 100
Validation prc-auc = 0.226382
Epoch 101
Validation prc-auc = 0.233868
Epoch 102
Validation prc-auc = 0.234147
Epoch 103
Validation prc-auc = 0.232578
Epoch 104
Loss = 5.8039e-03, PNorm = 85.7685, GNorm = 0.4751, lr_0 = 8.4730e-04
Validation prc-auc = 0.234665
Epoch 105
Validation prc-auc = 0.245773
Epoch 106
Validation prc-auc = 0.229765
Epoch 107
Validation prc-auc = 0.202668
Epoch 108
Validation prc-auc = 0.198487
Epoch 109
Loss = 3.3261e-03, PNorm = 85.8822, GNorm = 0.0878, lr_0 = 8.4006e-04
Validation prc-auc = 0.197556
Epoch 110
Validation prc-auc = 0.192250
Epoch 111
Validation prc-auc = 0.195907
Epoch 112
Validation prc-auc = 0.221345
Epoch 113
Validation prc-auc = 0.256502
Epoch 114
Loss = 4.8304e-03, PNorm = 85.9815, GNorm = 0.1888, lr_0 = 8.3288e-04
Validation prc-auc = 0.288530
Epoch 115
Validation prc-auc = 0.296416
Epoch 116
Validation prc-auc = 0.291004
Epoch 117
Validation prc-auc = 0.292835
Epoch 118
Validation prc-auc = 0.258954
Epoch 119
Loss = 1.6431e-03, PNorm = 86.0798, GNorm = 0.0257, lr_0 = 8.2576e-04
Validation prc-auc = 0.247764
Epoch 120
Validation prc-auc = 0.210904
Epoch 121
Validation prc-auc = 0.208080
Epoch 122
Validation prc-auc = 0.206742
Epoch 123
Validation prc-auc = 0.206940
Epoch 124
Loss = 9.5905e-04, PNorm = 86.1539, GNorm = 0.0384, lr_0 = 8.1870e-04
Validation prc-auc = 0.203504
Epoch 125
Validation prc-auc = 0.204715
Epoch 126
Validation prc-auc = 0.211334
Epoch 127
Validation prc-auc = 0.214867
Epoch 128
Validation prc-auc = 0.215314
Epoch 129
Loss = 4.8367e-04, PNorm = 86.2101, GNorm = 0.0060, lr_0 = 8.1170e-04
Validation prc-auc = 0.215197
Epoch 130
Validation prc-auc = 0.222110
Epoch 131
Validation prc-auc = 0.222112
Epoch 132
Validation prc-auc = 0.229797
Epoch 133
Validation prc-auc = 0.215422
Epoch 134
Loss = 4.2070e-03, PNorm = 86.2455, GNorm = 0.3350, lr_0 = 8.0476e-04
Validation prc-auc = 0.214897
Epoch 135
Validation prc-auc = 0.229929
Epoch 136
Validation prc-auc = 0.239647
Epoch 137
Validation prc-auc = 0.246353
Epoch 138
Validation prc-auc = 0.250734
Epoch 139
Loss = 6.0420e-04, PNorm = 86.3173, GNorm = 0.0343, lr_0 = 7.9788e-04
Validation prc-auc = 0.250647
Epoch 140
Validation prc-auc = 0.257317
Epoch 141
Validation prc-auc = 0.248711
Epoch 142
Validation prc-auc = 0.235555
Epoch 143
Validation prc-auc = 0.233822
Epoch 144
Loss = 6.8669e-04, PNorm = 86.3478, GNorm = 0.0313, lr_0 = 7.9106e-04
Validation prc-auc = 0.212841
Epoch 145
Validation prc-auc = 0.205824
Epoch 146
Validation prc-auc = 0.203542
Epoch 147
Validation prc-auc = 0.202880
Epoch 148
Validation prc-auc = 0.201224
Epoch 149
Loss = 4.4334e-04, PNorm = 86.4050, GNorm = 0.0098, lr_0 = 7.8430e-04
Validation prc-auc = 0.209669
Epoch 150
Validation prc-auc = 0.204278
Epoch 151
Validation prc-auc = 0.207228
Epoch 152
Validation prc-auc = 0.207041
Epoch 153
Validation prc-auc = 0.207041
Epoch 154
Loss = 3.5738e-04, PNorm = 86.4404, GNorm = 0.0020, lr_0 = 7.7759e-04
Validation prc-auc = 0.212598
Epoch 155
Validation prc-auc = 0.220758
Epoch 156
Validation prc-auc = 0.249183
Epoch 157
Validation prc-auc = 0.249469
Epoch 158
Validation prc-auc = 0.250900
Epoch 159
Loss = 9.2051e-04, PNorm = 86.4616, GNorm = 0.0203, lr_0 = 7.7095e-04
Validation prc-auc = 0.251085
Epoch 160
Validation prc-auc = 0.250245
Epoch 161
Validation prc-auc = 0.250634
Epoch 162
Validation prc-auc = 0.246110
Epoch 163
Validation prc-auc = 0.246093
Epoch 164
Loss = 3.2442e-04, PNorm = 86.4779, GNorm = 0.0125, lr_0 = 7.6436e-04
Validation prc-auc = 0.246118
Epoch 165
Validation prc-auc = 0.245354
Epoch 166
Validation prc-auc = 0.245917
Epoch 167
Validation prc-auc = 0.245151
Epoch 168
Validation prc-auc = 0.245481
Epoch 169
Loss = 1.9056e-04, PNorm = 86.4903, GNorm = 0.0037, lr_0 = 7.5782e-04
Validation prc-auc = 0.243320
Epoch 170
Validation prc-auc = 0.243397
Epoch 171
Validation prc-auc = 0.243613
Epoch 172
Validation prc-auc = 0.243942
Epoch 173
Validation prc-auc = 0.244307
Epoch 174
Loss = 4.2736e-04, PNorm = 86.5008, GNorm = 0.0080, lr_0 = 7.5134e-04
Validation prc-auc = 0.244400
Epoch 175
Validation prc-auc = 0.244494
Epoch 176
Validation prc-auc = 0.244454
Epoch 177
Validation prc-auc = 0.244431
Epoch 178
Validation prc-auc = 0.243393
Epoch 179
Loss = 2.6818e-04, PNorm = 86.5116, GNorm = 0.0115, lr_0 = 7.4492e-04
Validation prc-auc = 0.243267
Epoch 180
Validation prc-auc = 0.238144
Epoch 181
Validation prc-auc = 0.235473
Epoch 182
Validation prc-auc = 0.235404
Epoch 183
Validation prc-auc = 0.235608
Epoch 184
Loss = 1.8863e-04, PNorm = 86.5207, GNorm = 0.0080, lr_0 = 7.3855e-04
Validation prc-auc = 0.235624
Epoch 185
Validation prc-auc = 0.235521
Epoch 186
Validation prc-auc = 0.240956
Epoch 187
Validation prc-auc = 0.240956
Epoch 188
Validation prc-auc = 0.241229
Epoch 189
Loss = 2.0757e-04, PNorm = 86.5293, GNorm = 0.0066, lr_0 = 7.3224e-04
Validation prc-auc = 0.241229
Epoch 190
Validation prc-auc = 0.241110
Epoch 191
Validation prc-auc = 0.241110
Epoch 192
Validation prc-auc = 0.241110
Epoch 193
Validation prc-auc = 0.241021
Epoch 194
Loss = 6.9810e-05, PNorm = 86.5377, GNorm = 0.0019, lr_0 = 7.2598e-04
Validation prc-auc = 0.241021
Epoch 195
Validation prc-auc = 0.241140
Epoch 196
Validation prc-auc = 0.241040
Epoch 197
Validation prc-auc = 0.240974
Epoch 198
Validation prc-auc = 0.241132
Epoch 199
Loss = 6.9200e-05, PNorm = 86.5450, GNorm = 0.0011, lr_0 = 7.1977e-04
Validation prc-auc = 0.241248
Epoch 200
Validation prc-auc = 0.241248
Epoch 201
Validation prc-auc = 0.241436
Epoch 202
Validation prc-auc = 0.241436
Epoch 203
Validation prc-auc = 0.241436
Epoch 204
Loss = 1.2017e-04, PNorm = 86.5503, GNorm = 0.0028, lr_0 = 7.1362e-04
Validation prc-auc = 0.241436
Epoch 205
Validation prc-auc = 0.241436
Epoch 206
Validation prc-auc = 0.244757
Epoch 207
Validation prc-auc = 0.244883
Epoch 208
Validation prc-auc = 0.244796
Epoch 209
Loss = 6.0873e-05, PNorm = 86.5555, GNorm = 0.0016, lr_0 = 7.0752e-04
Validation prc-auc = 0.241373
Epoch 210
Validation prc-auc = 0.242334
Epoch 211
Validation prc-auc = 0.242443
Epoch 212
Validation prc-auc = 0.242608
Epoch 213
Validation prc-auc = 0.242856
Epoch 214
Loss = 1.3249e-04, PNorm = 86.5609, GNorm = 0.0044, lr_0 = 7.0147e-04
Validation prc-auc = 0.242639
Epoch 215
Validation prc-auc = 0.242574
Epoch 216
Validation prc-auc = 0.242574
Epoch 217
Validation prc-auc = 0.242574
Epoch 218
Validation prc-auc = 0.245897
Epoch 219
Loss = 5.6057e-05, PNorm = 86.5676, GNorm = 0.0017, lr_0 = 6.9548e-04
Validation prc-auc = 0.245913
Epoch 220
Validation prc-auc = 0.249503
Epoch 221
Validation prc-auc = 0.249583
Epoch 222
Validation prc-auc = 0.238346
Epoch 223
Validation prc-auc = 0.238017
Epoch 224
Loss = 7.1717e-05, PNorm = 86.5741, GNorm = 0.0008, lr_0 = 6.8953e-04
Validation prc-auc = 0.237636
Epoch 225
Validation prc-auc = 0.237557
Epoch 226
Validation prc-auc = 0.237557
Epoch 227
Validation prc-auc = 0.237557
Epoch 228
Validation prc-auc = 0.242161
Epoch 229
Loss = 7.9222e-05, PNorm = 86.5786, GNorm = 0.0057, lr_0 = 6.8364e-04
Validation prc-auc = 0.242221
Epoch 230
Validation prc-auc = 0.238465
Epoch 231
Validation prc-auc = 0.242052
Epoch 232
Validation prc-auc = 0.238603
Epoch 233
Validation prc-auc = 0.241255
Epoch 234
Loss = 4.5943e-05, PNorm = 86.5829, GNorm = 0.0010, lr_0 = 6.7779e-04
Validation prc-auc = 0.241878
Epoch 235
Validation prc-auc = 0.245466
Epoch 236
Validation prc-auc = 0.246428
Epoch 237
Validation prc-auc = 0.240818
Epoch 238
Validation prc-auc = 0.237349
Epoch 239
Loss = 3.8840e-05, PNorm = 86.5877, GNorm = 0.0011, lr_0 = 6.7200e-04
Validation prc-auc = 0.241673
Epoch 240
Validation prc-auc = 0.241791
Epoch 241
Validation prc-auc = 0.242033
Epoch 242
Validation prc-auc = 0.248715
Epoch 243
Validation prc-auc = 0.248478
Epoch 244
Loss = 3.0307e-05, PNorm = 86.5913, GNorm = 0.0014, lr_0 = 6.6625e-04
Validation prc-auc = 0.247657
Epoch 245
Validation prc-auc = 0.258528
Epoch 246
Validation prc-auc = 0.247285
Epoch 247
Validation prc-auc = 0.245310
Epoch 248
Validation prc-auc = 0.245393
Epoch 249
Loss = 2.7917e-05, PNorm = 86.5958, GNorm = 0.0006, lr_0 = 6.6056e-04
Validation prc-auc = 0.239596
Epoch 250
Validation prc-auc = 0.245767
Epoch 251
Validation prc-auc = 0.245592
Epoch 252
Validation prc-auc = 0.245608
Epoch 253
Validation prc-auc = 0.251031
Epoch 254
Loss = 3.1287e-05, PNorm = 86.6002, GNorm = 0.0007, lr_0 = 6.5491e-04
Validation prc-auc = 0.251048
Epoch 255
Validation prc-auc = 0.250844
Epoch 256
Validation prc-auc = 0.254301
Epoch 257
Validation prc-auc = 0.257112
Epoch 258
Validation prc-auc = 0.256776
Epoch 259
Loss = 4.4208e-05, PNorm = 86.6044, GNorm = 0.0050, lr_0 = 6.4931e-04
Validation prc-auc = 0.255416
Epoch 260
Validation prc-auc = 0.250021
Epoch 261
Validation prc-auc = 0.248634
Epoch 262
Validation prc-auc = 0.245542
Epoch 263
Validation prc-auc = 0.245542
Epoch 264
Loss = 3.8877e-05, PNorm = 86.6086, GNorm = 0.0003, lr_0 = 6.4376e-04
Validation prc-auc = 0.245999
Epoch 265
Validation prc-auc = 0.245936
Epoch 266
Validation prc-auc = 0.247678
Epoch 267
Validation prc-auc = 0.251398
Epoch 268
Validation prc-auc = 0.269290
Epoch 269
Loss = 2.9516e-05, PNorm = 86.6109, GNorm = 0.0013, lr_0 = 6.3826e-04
Validation prc-auc = 0.269625
Epoch 270
Validation prc-auc = 0.269625
Epoch 271
Validation prc-auc = 0.269516
Epoch 272
Validation prc-auc = 0.266489
Epoch 273
Validation prc-auc = 0.255238
Epoch 274
Loss = 2.3526e-05, PNorm = 86.6148, GNorm = 0.0008, lr_0 = 6.3280e-04
Validation prc-auc = 0.248307
Epoch 275
Validation prc-auc = 0.248124
Epoch 276
Validation prc-auc = 0.248124
Epoch 277
Validation prc-auc = 0.247711
Epoch 278
Validation prc-auc = 0.247905
Epoch 279
Loss = 1.7578e-05, PNorm = 86.6185, GNorm = 0.0015, lr_0 = 6.2739e-04
Validation prc-auc = 0.247873
Epoch 280
Validation prc-auc = 0.247758
Epoch 281
Validation prc-auc = 0.247821
Epoch 282
Validation prc-auc = 0.247629
Epoch 283
Validation prc-auc = 0.247742
Epoch 284
Loss = 2.6666e-05, PNorm = 86.6215, GNorm = 0.0014, lr_0 = 6.2203e-04
Validation prc-auc = 0.255118
Epoch 285
Validation prc-auc = 0.256661
Epoch 286
Validation prc-auc = 0.256779
Epoch 287
Validation prc-auc = 0.259250
Epoch 288
Validation prc-auc = 0.259301
Epoch 289
Loss = 1.7281e-05, PNorm = 86.6252, GNorm = 0.0011, lr_0 = 6.1671e-04
Validation prc-auc = 0.256731
Epoch 290
Validation prc-auc = 0.255539
Epoch 291
Validation prc-auc = 0.255066
Epoch 292
Validation prc-auc = 0.254954
Epoch 293
Validation prc-auc = 0.255534
Epoch 294
Loss = 4.2097e-05, PNorm = 86.6282, GNorm = 0.0024, lr_0 = 6.1144e-04
Validation prc-auc = 0.255490
Epoch 295
Validation prc-auc = 0.255597
Epoch 296
Validation prc-auc = 0.253429
Epoch 297
Validation prc-auc = 0.250955
Epoch 298
Validation prc-auc = 0.249664
Epoch 299
Loss = 2.6413e-05, PNorm = 86.6322, GNorm = 0.0020, lr_0 = 6.0621e-04
Validation prc-auc = 0.247300
Model 0 best validation prc-auc = 0.497692 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test prc-auc = 0.401629
Ensemble test prc-auc = 0.401629
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.402364
	Seed 1 ==> test prc-auc = 0.387643
	Seed 2 ==> test prc-auc = 0.473718
	Seed 3 ==> test prc-auc = 0.528662
	Seed 4 ==> test prc-auc = 0.426883
	Seed 5 ==> test prc-auc = 0.387804
	Seed 6 ==> test prc-auc = 0.381926
	Seed 7 ==> test prc-auc = 0.512151
	Seed 8 ==> test prc-auc = 0.437878
	Seed 9 ==> test prc-auc = 0.401629
Overall test prc-auc = 0.434066 +/- 0.050750
Elapsed time = 0:08:52
